What is the most cost-efficient form of compute to buy?

Message boards : Questions and problems : What is the most cost-efficient form of compute to buy?
Message board moderation

To post messages, you must log in.

AuthorMessage
bwindsor22

Send message
Joined: 4 Sep 20
Posts: 20
United States
Message 112026 - Posted: 8 Jun 2023, 3:10:02 UTC

Hi all, I'm thinking about adding some additional compute to my setup, and wondering if anyone has done the math on what is the cheapest way to gain additional computing power.

For instance, I see a range of possible purchase options:
* Old desktops on Ebay, 2-6 cores, $100-500
* Mini PCs, 1-8 cores, $250-800
* Gaming PCs, ~6 cores, with an NVidia GPU, $1,000-$3,000
* Workstation PCs/mini servers, ~24 cores but no GPU, $2,000-4,000

There are several tradeoffs in these choices:
* Buy more of smaller machines, vs buy fewer of larger ones
* Buy new hardware which costs more up front but uses less electricity in the long run, vs buy cheap hardware initially
* Spend more $ for GPUs, or spend it on CPUs

What do you think? Seems like a giant linear optimization problem. What would you say is the best bang for your buck when adding new compute?[/list]
ID: 112026 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2533
United Kingdom
Message 112030 - Posted: 8 Jun 2023, 5:26:32 UTC - in response to Message 112026.  

Hi all, I'm thinking about adding some additional compute to my setup, and wondering if anyone has done the math on what is the cheapest way to gain additional computing power.

For instance, I see a range of possible purchase options:
* Old desktops on Ebay, 2-6 cores, $100-500
* Mini PCs, 1-8 cores, $250-800
* Gaming PCs, ~6 cores, with an NVidia GPU, $1,000-$3,000
* Workstation PCs/mini servers, ~24 cores but no GPU, $2,000-4,000

There are several tradeoffs in these choices:
* Buy more of smaller machines, vs buy fewer of larger ones
* Buy new hardware which costs more up front but uses less electricity in the long run, vs buy cheap hardware initially
* Spend more $ for GPUs, or spend it on CPUs

What do you think? Seems like a giant linear optimization problem. What would you say is the best bang for your buck when adding new compute?[/list]
I think you are looking at things the wrong way round. First thing to do is decide which projects you are interested in and see what the requirements are for those projects. Some projects have in their portfolio tasks that run partly or mostly on video cards which brings another variable into the equation. Will the machine(s) only be used for crunching? If not what else? Once yo have decided on which projects, it may be worth looking on the forums for those projects to see what users there think.
ID: 112030 · Report as offensive
Profile Keith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 867
United States
Message 112031 - Posted: 8 Jun 2023, 7:45:02 UTC

The best optimization comes from running your BOINC projects, 24/7. But with that requirement, you have to figure in your power costs. Can you afford to run your cruncher 24/7, every hour, every day, every month. If you have high energy tariffs, you need to think about how much power you are willing to pay for.

In this matter, the small Soc and/or mini-PC comes to the fore. A lot of the newer mini-PC come with new core and graphics architectures. You can get up to 8c/16t and a embedded gpu that is able to crunch some gpu tasks at projects as long as they have applications.

You will not be able to match the power thriftyness of the SoC or mini-PC compared to a standard PC of older generation. It will also produce the lesser heat output that you have to deal with enviromentally in your location.

If you have very low tariff costs, then the older PC with an 1-2 generation older gpu would provide good output and produce higher output compared to the mini-PC. The heat output of this type of cruncher probably will require air conditioning if used indoors where you live. That adds to the power cost of running the system.
ID: 112031 · Report as offensive
bwindsor22

Send message
Joined: 4 Sep 20
Posts: 20
United States
Message 112050 - Posted: 11 Jun 2023, 14:31:44 UTC

Thank you! Good comments both, great getting the advice from the pros here.

Yes, I suppose you're right, probably better to optimize by the project's needs.

And mini pcs were what I was leaning towards too -- they seem to offer far more cores per $ than the big workstation PCs, which I found unintuitive. I would have guessed there were economies of scale (cost per cpu goes down when you share components like the motherboard), but online I'm finding the opposite-- a $400-600 Mini PC gives me ~15 cores per $1000 spent, while anything from the server/workstation category give me <=5 cores per $1000. Wattage is hard to calculate.

So anything larger than a mini PC is a waste of money? Maybe it's that bigger machines make up for their higher cost/core with stronger chips? When I look at the top users, it does seem that anyone who breaks the top ~1000 is running servers, e.g.:
https://www.boincstats.com/stats/-1/host/list/0/0/66b4359476520659b8db798e6d933131
https://www.boincstats.com/stats/-1/host/list/0/0/1af584e3ffa95fd941d2d232fab98611

Here's my compute vs cost calculation:

Name Count Cpu Cost Cores / $1000
Beelink Mini PC, AMD 8 $412.64 19.39
KAMRUI AMD Ryzen 7 3 4 $296.65 13.48
MINISFORUM UM690 Min 8 $631.99 12.66
Beelink GTR6 Mini PC 8 $691.34 11.57
Lenovo IdeaCentre 5 12 $1,089.00 11.02
HP Z2 G9 Tower Works 24 $2,992.00 8.02
Lenovo ThinkCentre M 1 $181.52 5.51
OMEN 25L Gaming Desk 8 $1,477.99 5.41
Titan S475 - Dual AM 64 $16,732.00 3.83
Hewlett Packard Ente 3 $1,009.00 2.97
Hpe ProLiant ML350 G 10 $4,409.99 2.27

https://docs.google.com/spreadsheets/d/1oTH_pctZzEnrgjiLow6jFyLJsl9jcBdemTAzX8eoNbg/edit?usp=sharing
ID: 112050 · Report as offensive
robsmith
Volunteer tester
Help desk expert

Send message
Joined: 25 May 09
Posts: 1283
United Kingdom
Message 112054 - Posted: 11 Jun 2023, 16:29:47 UTC

Don't get too hung up on cost per core because there are a lot more things that have to be taken into consideration - here's a few more things that come into play:
RAM - the more cores the more RAM you will need - somewhere between 1Gb and 4Gb per core;
SSD vs. HDD - you will need something (at least one)
GPUs - depending on the projects you are going to run a reasonable GPU can easily out perform a high core count
Then there's electricity - generally the bigger the core count, the more RAM and the more GPUs the more the computer will use
Then, if you live on a metered internet connection can become a consideration.

Then the mix of projects you run cuts in......
ID: 112054 · Report as offensive
Profile Keith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 867
United States
Message 112055 - Posted: 11 Jun 2023, 16:33:08 UTC - in response to Message 112050.  

The reason that servers lead the stats is simply because of scale. They have far more cores than even the best mini-PC. Hundreds of cores vs 8-16 in the mini-PCs.

Plus the servers are able to use multiple gpus which are more productive than cpu cores. But the cost efficiency is still probably in the favor of the mini-PC because server hardware, memory and multiple gpus cost 10X- 100X more than the typical mini-PC. They also use far more power which doubles, triples or quadruples the power cost of running a crunching computer on BOINC.
ID: 112055 · Report as offensive
HAL9000
Help desk expert
Avatar

Send message
Joined: 13 Jun 14
Posts: 81
United States
Message 112126 - Posted: 25 Jun 2023, 9:06:07 UTC

When I first got my Abit il-90mv I have dreamt up plans of have a huge array of multiple mini machines to do crunching.
Since then and having runs lab with 50+ machines I have decided that managing loads of machines is kind of a pain.
Although I would still like to do a smaller scale version of what I first cooked up in my head using small boards like my ASRock J5040-ITX.
Ideally I would like to get 8-12, stick them in a cabinet with pico PSUs to run off of a single 12v supply, and a few 200mm fans to cool them.
Currently I am just holding out to try and find a good board like the J5040-ITX that has a PCIe SSD slot.
I did just order a Beelink EQ12 N305 to play around with a bit, but it is a full mini pc rather than just a motherboard with a SoC. I am expecting the little fan in it to sound like a jet taking off when I run all 8 cores full tilt.

A few years ago I did get some old retired dual Xeon boards with two E5-2690v2 CPUs and 128GB of ram. Which were about $400.
Buying the power supply, case, and other bits to complete them I may have spent $700 total for each.
After 7 years they are still going along quite well, and even compared to my new i5-13600KF they still do quite well crunching tasks.
Currently I am starting to look to replace their GTX 750 Ti GPUs since graphic cards new exist again for somewhat reasonable prices.
ID: 112126 · Report as offensive

Message boards : Questions and problems : What is the most cost-efficient form of compute to buy?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.