Best Mining Motherboards for BOINC?

Message boards : The Lounge : Best Mining Motherboards for BOINC?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Tom Miller

Send message
Joined: 6 Jul 14
Posts: 67
United States
Message 95439 - Posted: 22 Jan 2020, 12:55:08 UTC

Hi,
I wanted to start this thread here because it has been fruitful on the Seti@Home forum.

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
An optimist is POSITIVE the glass is half-full.
In detail I am a Big Picture sort of guy.
ID: 95439 · Report as offensive     Reply Quote
Profile Tom Miller

Send message
Joined: 6 Jul 14
Posts: 67
United States
Message 95441 - Posted: 22 Jan 2020, 12:56:27 UTC

I am looking for (in the near future) an easy to setup mining rack for 18 or more Gpus.

Do you have any opinions? First hand experience?

Thank you.

Tom
ID: 95441 · Report as offensive     Reply Quote
Profile Tom Miller

Send message
Joined: 6 Jul 14
Posts: 67
United States
Message 95443 - Posted: 22 Jan 2020, 13:03:59 UTC

https://www.biostar.com.tw/app/en/mb/introduction.php?S_ID=882
I have had experience with a Biostar tb350-btc. It has 6 slots and when the Moon is correctly aligned it will run as many as 9 gpus at once.

I have a couple of ultra-stable ASUS MB's (a Cross Hair Hero VII and a Prime...) and they won't run more than 6 gpus.

I have also played with a couple of Intel server motherboards and was able to get 7-9 gpus running on 5-7 slot boards.

My next experiment will be with a "MSI B360-F PRO LGA 1151 Intel B360 ATX (7B25-001R)" which is a version 2 of that socket that will let me run cpus with 8c/16t's. The MB has 18 slots.

An analysis by another Setizen is there are bottlenecks in the board design that won't let me run 18 gpus at full Seti@Home speed.

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
An optimist is POSITIVE the glass is half-full.
In detail I am a Big Picture sort of guy.
ID: 95443 · Report as offensive     Reply Quote
ProDigit

Send message
Joined: 8 Nov 19
Posts: 460
United States
Message 95575 - Posted: 29 Jan 2020, 1:53:59 UTC
Last modified: 29 Jan 2020, 2:23:02 UTC

First of all,
You'd have to consider power consumption, running cost, and cooling.
Does your home have sufficient power lines to feed all these GPUs?
In the US, on most lines you can safely add 1400W of continuous load.
That means 2800W if you have access to a dual phase, 220V circuit.
In Europe, it's a little higher (3000W per phase for continuous load).

Power consumption, would mean cooling capabilities of at least equal. 2000W of servers, needs at least a 2000W (~10.000 BTU) AC system, to keep everything cool, if the air is recirculating.
Perhaps a heat pump might be a more economical solution.
Or some powerful fannery to blast the heat away from the servers (like, suck in outside air, and blow it outside).

Also, 2000W system costs about $2k a year to keep running. $4k if you include AC units.

Second, it pays you back, to go for 1 fast GPU over ten slower GPUs.
A single RTX 2070 Super is about twice as expensive, as an RTX 2060, and performs only 1,5x the work.
But it can be made run reliably at 150W, vs 125W for the 2060. That means, in a system of 3x RTX 2060 vs 2x 2070, you'll run the same performance, but every year you'll save $50 on electricity. The initial purchase price difference, will be paid back in a few years on electricity.
The difference is much greater with Pascal (GTX) GPUs vs Turing (RTX) GPUs, since Pascal GPUs are still sold pretty much at new prices (their prices haven't really dropped, and their performance and efficiency is a lot lower than Turing GPUs).
If your power lines will provide all the power you need, then GTX 1060s would be great affordable GPUs or even AMD GPUs would work nice, but most people run a power ceiling at some point, at which you have no other option than to go to the more expensive GPUs.
Then it makes more sense to go expensive from the beginning, and buy a set of RTX 2080 Tis, and add a new GPU whenever the funds are there.

Depending on project to project, most projects will expect your GPU to be at GTX 1060 speeds, or faster.
Compared to a 1060, a 2080 Ti runs 3x more cores, and thus can be said to be 3x faster for some projects; while running at 180W vs 3x ~95W on the 1060s as most optimal setting.
As you may notice, I run mine power capped, just to get the highest efficiency out of them.
But even at full power draw of 110W vs 250-275W on a stock setting, the 2080 Ti still wins.

Third, to have that many GPUs on a board, you'll need many CPU cores (which you seem to have).
Most mining boards work fine from quadcore CPUs, or even dual core celerons, because mining takes very little pcie data traffic and cpu overhead.
Boinc is different.
For each RTX GPU, you'll need about a 3Ghz single (non hyperthreaded) CPU core. For a GT 1030-GTX 1060 GPU you'll need at least a 2,5Ghz CPU (to have sufficient CPU power to feed the GPU).
Many Xeon CPUs run at lower speeds (<3Ghz) and wouldn't be very good for folding.

Fourth, PCIE lanes.
If your PCIE lanes are 3.0 x1 slots, they are good enough for RTX 2060 Super to 2070 GPUs.
If they're PCIE 2.0 x1 slots, they're only good for up to 1050 GPUs.
While you could run faster GPUs in them, like an RTX GPU (provided your Bios supports them), they will be running at only a fraction of their potential.

Most people would run either a cheaper 10 GPU board with a cheaper 4C/8T CPU, at lower efficiencies,
Or run more PCIE 3.0 motherboards with quadcore CPUs and higher end RTX GPUs at much higher efficiencies.
It's up to you to decide what's best.
Most motherboards are only able to push up to 3x RTX GPUs at full speed, and one additional PCIE 1x slot (for slower RTX 2060 GPUs).

Even power capped, 4 RTX GPUs run at well above 800 Watts, just barely what most 1000W PSUs can efficiently handle.

My approach is to use low power CPU's like a Pentium Gold (2C/4T), or Core I3 (4C) to feed 3-4 high end GPUs on cheap $100 motherboards; rather than expensive multicore CPUs pushing multi GPUs over inefficient and slow PCIE lanes.
The cons of my setup is that I get 80-90W overhead on each 3GPU system (90W divided over CPU, board hardware, and PSU efficiencies).
Something you won't have to worry about with a mining setup; but then again, a mining setup runs much less efficient.
ID: 95575 · Report as offensive     Reply Quote
Profile Tom Miller

Send message
Joined: 6 Jul 14
Posts: 67
United States
Message 96123 - Posted: 26 Feb 2020, 2:34:36 UTC - in response to Message 95575.  


My approach is to use low power CPU's like a Pentium Gold (2C/4T), or Core I3 (4C) to feed 3-4 high end GPUs on cheap $100 motherboards; rather than expensive multicore CPUs pushing multi GPUs over inefficient and slow PCIE lanes.
The cons of my setup is that I get 80-90W overhead on each 3GPU system (90W divided over CPU, board hardware, and PSU efficiencies).
Something you won't have to worry about with a mining setup; but then again, a mining setup runs much less efficient.


Thank you for your extensive and thoughtful reply.

With the Linux + the Tbar/Petri AIO I have been able to run the gpus at the default 0.1 cpu. So far the wall clock time and amount of CPU usage are tracking pretty closely.

I was worried about having enough cpu threads to actually drive up to 18 gpus which is why I bought an 8th-9th Gen MB and a 8c/16t cpu.

I have two 20 Amp circuits and a 15 Amp circuit that I will be splitting the load across. One 20 Amp circuit is completely unused so I am running it as my primary line now (with 10 gpus). I am going to have to get some 20 Amp rated surge-protectors. My current ones are overloading. Unfortunately the other circuits have more than the Seti box drawing on them. So it simply may not be possible to provide enough power to run all 18.

But wait, there's more :) I am going to have to buy a 19 gpu mining frame and longer UGREEN 3.0USB cables as well as 20 Amp surge-protectors before I can experiment with the higher end stuff.

I may have to rob some gpu's from some other systems for a trial.

I do have 1 RTX 2060 Super which according to a regularly updated report by a guy named "Shaggie" comes out with the best performance/power efficiency combo.

Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
An optimist is POSITIVE the glass is half-full.
In detail I am a Big Picture sort of guy.
ID: 96123 · Report as offensive     Reply Quote
Profile Joseph Stateson
Volunteer tester
Avatar

Send message
Joined: 27 Jun 08
Posts: 538
United States
Message 96704 - Posted: 12 Mar 2020, 16:24:00 UTC

Got to looking at a 7 slot PCIe server: dual xeon 1366 for mining. multicore xeon's are considerably cheaper than any comparable gen 6 or gen 7.
https://www.ebay.com/itm/Xyratex-0944037-02-Dual-Socket-1366-Server-System-Motherboard/392343147646?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649

Worked fine with Linux although I did have a problem with the AMD driver but that was due to the GPUs being unusual: "s9050" Windows 7 and 8.x worked fine with those low power HD7950 equivalents.
Currently have 8.1 installed only because I had an unused license.

DDR3 ECC server memory is cheap. The mombo is strictly for mining as no x16 slots and is not ATX or EATX size. Screw holes do not line up even with the mining rack.
Power supply requires an extra 8 pin but otherwise uses standard ATX power some of which have the extra 8pin.
It can do both Gen2 and Gen1 but not Gen3

I looked at inserting a license into the bios but didn't try. If one of my TB85 boards crap out I may get another and try my bios mod.
https://www.bios-mods.com/forum/Thread-xyrantex-two-different-bios-onboard-mombo
ID: 96704 · Report as offensive     Reply Quote
ProDigit

Send message
Joined: 8 Nov 19
Posts: 460
United States
Message 96725 - Posted: 13 Mar 2020, 5:32:40 UTC - in response to Message 96123.  
Last modified: 13 Mar 2020, 5:36:54 UTC


My approach is to use low power CPU's like a Pentium Gold (2C/4T), or Core I3 (4C) to feed 3-4 high end GPUs on cheap $100 motherboards; rather than expensive multicore CPUs pushing multi GPUs over inefficient and slow PCIE lanes.
The cons of my setup is that I get 80-90W overhead on each 3GPU system (90W divided over CPU, board hardware, and PSU efficiencies).
Something you won't have to worry about with a mining setup; but then again, a mining setup runs much less efficient.


Thank you for your extensive and thoughtful reply.

With the Linux + the Tbar/Petri AIO I have been able to run the gpus at the default 0.1 cpu. So far the wall clock time and amount of CPU usage are tracking pretty closely.

I was worried about having enough cpu threads to actually drive up to 18 gpus which is why I bought an 8th-9th Gen MB and a 8c/16t cpu.

I have two 20 Amp circuits and a 15 Amp circuit that I will be splitting the load across. One 20 Amp circuit is completely unused so I am running it as my primary line now (with 10 gpus). I am going to have to get some 20 Amp rated surge-protectors. My current ones are overloading. Unfortunately the other circuits have more than the Seti box drawing on them. So it simply may not be possible to provide enough power to run all 18.

But wait, there's more :) I am going to have to buy a 19 gpu mining frame and longer UGREEN 3.0USB cables as well as 20 Amp surge-protectors before I can experiment with the higher end stuff.

I may have to rob some gpu's from some other systems for a trial.

I do have 1 RTX 2060 Super which according to a regularly updated report by a guy named "Shaggie" comes out with the best performance/power efficiency combo.

Tom


I think I was the one recommending this GPU about 1 to 2 years ago all over the Internet.
The landscape has changed, though not by much.
The 2060 Super is currently the better deal. And in some cases a 2070 Super.
In my case, where I am running at my power ceiling, I ONLY run RTX2080Ti GPUs. (I can't justify the RTX Titans (with only 15-20% higher performance) costing twice the price.

I recently have added a Ryzen 9 3900x to my setup, to do CPU crunching for a variety of smaller projects that have no GPU support.
The Ryzen is pretty awesome! I've had to swap out the stock cooler, because that one was pure garbage, and would overheat the CPU.
I also needed to lower the Ryzen CPU to 65W instead (AMD's ECO MODE) of the rated 105W TDP.
But even at 65W with a $160 watercooling system, it runs 24 threads at 3,9Ghz (vs stock cooler in eco mode at 3,1Ghz)!
I tried 4Ghz in eco mode on the water cooler, but ran into some issues, and had to dial back down to 3,9Ghz.
Running it stock at 105W with water cooling, I could possibly boost the speed on all cores to just under 4,2Ghz.
But considering that CPU could run between 105 and 160W when overclocked, I prefer the lower power bill, and distribute the remaining (40+ watts of) power evenly to all my GPUs.

The Ryzen 9 3900x came out as the best deal for me. The 3950 is just a tad faster, but nearly twice as expensive.
The next step above that, is a 16+ core 3rd gen threadripper; however, I couldn't justify the $1700.
I'll run the 3900x for another year, and upgrade as the multicores become more affordable!
ID: 96725 · Report as offensive     Reply Quote

Message boards : The Lounge : Best Mining Motherboards for BOINC?

Copyright © 2020 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.