I'd noticed many comments in various fora that the Linux build of BOINC has lower benchmarks than the Windows build, often accompanied by specific data for one system. I wanted a better overall view of the difference, and having achieved that I'm posting it here as possibly interesting information.

I used the Einstein@home Feb. 4 host_id file since I'm on dial-up, it only took 1 1/2 hours to download. Then I extracted the benchmark data for p_models which included numbers to indicate clock rate and which had at least 20 Linux hosts and 20 Windows hosts. That left me with 81 CPUs representing 117812 hosts.

For each p_model I calculated separate averages for the Linux and Windows benchmarks. I discarded the lowest and highest 15% and used the arithmetic mean of the 70% remaining as the average. The discarding should have eliminated most if not all hosts which run an optimized BOINC build, were affected by other running applications, etc.

Here's the visual overview of the results (the vertical scale is Whetstone MIPS + Dhrystone MIPS, the "reference computer" would be at 2000):



A raw average of the 81 Windows/Linux ratios is 1.543, an average weighted by the number of hosts using each CPU is 1.514.

The data set below is the basis for the image.

Lin. -- Win. -- hosts -- p_model
1366 - 2113 - 00381 - Intel(R) Pentium(R) 4 CPU 1400MHz
1460 - 2240 - 00700 - Intel(R) Pentium(R) 4 CPU 1500MHz
1500 - 2269 - 00422 - Intel(R) Pentium(R) 4 CPU 1.50GHz
1907 - 2400 - 00576 - Intel(R) Xeon(TM) CPU 2.40GHz
1734 - 2441 - 00133 - Intel(R) Xeon(TM) CPU 2.00GHz
1554 - 2451 - 01637 - Intel(R) Pentium(R) 4 CPU 1.60GHz
1670 - 2543 - 00496 - Intel(R) Pentium(R) 4 CPU 1700MHz
1750 - 2554 - 00926 - Intel(R) Celeron(R) CPU 1.70GHz
1667 - 2583 - 01342 - Intel(R) Pentium(R) 4 CPU 1.70GHz
1972 - 2710 - 02637 - Intel(R) Pentium(R) 4 CPU 2.60GHz
1800 - 2745 - 02706 - Intel(R) Pentium(R) 4 CPU 1.80GHz
1796 - 2771 - 00431 - Intel(R) Celeron(R) CPU 1.80GHz
1643 - 2801 - 00121 - Intel(R) Pentium(R) III CPU family 1133MHz
1928 - 2838 - 00846 - Intel(R) Xeon(TM) CPU 2.80GHz
2934 - 2907 - 00481 - Intel(R) Xeon(TM) CPU 3.06GHz
2059 - 2952 - 12781 - Intel(R) Pentium(R) 4 CPU 3.00GHz
1850 - 2976 - 00162 - Intel(R) Xeon(TM) CPU 2.66GHz
1805 - 2985 - 00476 - Intel(R) Celeron(TM) CPU 1200MHz
2679 - 2996 - 01314 - Intel(R) Pentium(R) 4 CPU 3.06GHz
2095 - 3014 - 00334 - Intel(R) Xeon(TM) CPU 3.00GHz
1950 - 3026 - 03357 - Intel(R) Pentium(R) 4 CPU 2.00GHz
2092 - 3074 - 00969 - Intel(R) Celeron(R) CPU 2.00GHz
1904 - 3145 - 00134 - Intel(R) Pentium(R) III CPU family 1266MHz
2361 - 3156 - 00441 - Intel(R) Xeon(TM) CPU 3.20GHz
2236 - 3163 - 05712 - Intel(R) Pentium(R) 4 CPU 3.20GHz
1928 - 3208 - 00348 - Intel(R) Celeron(TM) CPU 1300MHz
2260 - 3239 - 00426 - Intel(R) Celeron(R) CPU 2.53GHz
2311 - 3253 - 11113 - Intel(R) Pentium(R) 4 CPU 2.80GHz
2437 - 3392 - 00660 - Intel(R) Celeron(R) CPU 2.66GHz
2146 - 3430 - 00098 - AMD Athlon(TM) XP 1600+
2162 - 3431 - 00677 - AMD Athlon(tm) XP 1600+
2260 - 3451 - 00193 - Intel(R) Xeon(TM) CPU 3.40GHz
2604 - 3453 - 02435 - Intel(R) Pentium(R) 4 CPU 3.40GHz
2257 - 3488 - 00746 - Intel(R) Pentium(R) 4 CPU 2.26GHz
2446 - 3547 - 02174 - Intel(R) Celeron(R) CPU 2.40GHz
2368 - 3589 - 08593 - Intel(R) Pentium(R) 4 CPU 2.40GHz
2364 - 3599 - 01118 - AMD Athlon(tm) XP 1700+
2552 - 3601 - 00145 - Intel(R) Xeon(TM) CPU 3.60GHz
2271 - 3636 - 00228 - AMD Sempron(tm) - 2200+
2319 - 3740 - 01119 - Intel(R) Pentium(R) M processor 1400MHz
2394 - 3761 - 02036 - AMD Athlon(tm) XP 1800+
2687 - 3836 - 00850 - Intel(R) Celeron(R) CPU 2.80GHz
2561 - 3878 - 00701 - Intel(R) Celeron(R) CPU 2.60GHz
2593 - 3944 - 01947 - Intel(R) Pentium(R) 4 CPU 2.53GHz
2330 - 3984 - 00994 - Intel(R) Pentium(R) M processor 1500MHz
2356 - 4010 - 00423 - AMD Athlon(tm) XP 1900+
2924 - 4015 - 00079 - AMD Athlon(tm) MP 2000+
2438 - 4074 - 00283 - AMD Sempron(tm) - 2400+
2721 - 4080 - 03022 - Intel(R) Pentium(R) 4 CPU 2.66GHz
2308 - 4130 - 00480 - AMD Athlon(TM) XP 2000+
2565 - 4131 - 02455 - AMD Athlon(tm) XP 2000+
2629 - 4164 - 00067 - AMD Opteron(tm) Processor 242
2619 - 4206 - 01640 - Intel(R) Pentium(R) M processor 1.60GHz
2746 - 4217 - 01138 - Intel(R) Pentium(R) M processor 1600MHz
3103 - 4257 - 00102 - AMD Athlon(tm) MP 2200+
2825 - 4297 - 00599 - AMD Athlon(tm) XP 2100+
2777 - 4421 - 01650 - AMD Athlon(tm) XP 2200+
2629 - 4455 - 00262 - AMD Athlon(TM) XP 2500+
2861 - 4474 - 00343 - AMD Athlon(TM) XP 2200+
2576 - 4488 - 00911 - Intel(R) Pentium(R) M processor 1.70GHz
2855 - 4575 - 02508 - AMD Athlon(tm) XP 2500+
3021 - 4646 - 00410 - AMD Athlon(TM) XP 2400+
2900 - 4828 - 00694 - AMD Athlon(tm) 64 Processor 2800+
3112 - 4861 - 00317 - AMD Athlon(TM) XP 2600+
3157 - 4922 - 02623 - AMD Athlon(tm) XP 2400+
3137 - 4938 - 02890 - AMD Athlon(tm) XP 2600+
2824 - 4955 - 00072 - AMD Athlon(tm) MP 2600+
3143 - 4974 - 00080 - AMD Opteron(tm) Processor 244
3271 - 5178 - 02086 - AMD Athlon(tm) XP 2800+
3064 - 5184 - 04088 - AMD Athlon(tm) 64 Processor 3000+
3184 - 5263 - 00776 - Intel(R) Pentium(R) M processor 2.00GHz
3104 - 5276 - 00081 - AMD Opteron(tm) Processor 246
3392 - 5311 - 02029 - AMD Athlon(tm) XP 3000+
3309 - 5445 - 00552 - AMD Athlon(tm) XP 2700+
3515 - 5473 - 01799 - AMD Athlon(tm) XP 3200+
3031 - 5477 - 00490 - AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
3431 - 5533 - 03528 - AMD Athlon(tm) 64 Processor 3200+
3456 - 5910 - 01911 - AMD Athlon(tm) 64 Processor 3500+
3157 - 5943 - 00108 - AMD Opteron(tm) Processor 248
3764 - 6257 - 01154 - AMD Athlon(tm) 64 Processor 3400+
3898 - 6461 - 00045 - AMD Opteron(tm) Processor 250

                                                       Joe

It is funny you have them sorted by the Windows column and the Celeron is doing better than a lot of the other chips. I think you need to break apart the FPiops and the Iops and compare separately. If you sort by Linux, the order looks better. If the method you used is sound, I would have to wonder about the validity of the benchmarks in Windows. I was playing the stats too but just SPARC/Solaris and there aren't enough to make a good conclusion on but the benchmarks look funny there too. I was looking at the comparison of a Sunblade 100 500 Mhz to a Sunfire 2500 1.2 Ghz and the SPEC benchmarks. I expected a lot more of a difference between the models but it doesn't show in the Boinc benchmarks.

---

It is funny you have them sorted by the Windows column and the Celeron is doing better than a lot of the other chips. I think you need to break apart the FPiops and the Iops and compare separately. If you sort by Linux, the order looks better. If the method you used is sound, I would have to wonder about the validity of the benchmarks in Windows. I was playing the stats too but just SPARC/Solaris and there aren't enough to make a good conclusion on but the benchmarks look funny there too. I was looking at the comparison of a Sunblade 100 500 Mhz to a Sunfire 2500 1.2 Ghz and the SPEC benchmarks. I expected a lot more of a difference between the models but it doesn't show in the Boinc benchmarks.

The reason Celerons do well is that both Whetstone and Dhrystone benchmarks run in L1 cache. That's just one of the unrealistic aspects, of course.

Whether the Windows benchmarks are more or less valid than other platforms is effectively moot, over 88% of the hosts running BOINC are Windows systems so their benchmarks become the standard.

One of the reasons I provided the data listing is so others could look at it for other purposes.

Perhaps the only general conclusion which can be drawn is that all projects should switch to a method of calculating credit claims which doesn't include the BOINC benchmarks.

                                                     Joe

Interestingly both 3.06 GHz Linux models (Xeon and Pentium 4) jump out of line and are (nearly) equal with their Windows counterparts.

Peter

On the basis of one of the 5.8 target development items being a comparable Linux-Windows benchmark i wonder which BOINC versions formed the basis of this data collection.

Good question, Sekerob. As Joe Segur used the host_id file, I assume he have had no opportunity to check the Boinc versions...

Maybe now few weeks ago (just before new recommended Boinc version 5.8.x) would be suitable time to du such "survey" once again. It was long long time since the previous version become the recommended one.

Peter

On my 3.0GHz P4 Prescott oc'd to 3.37GHz I get the following FLOPs/IOPs.

1. Linux - FC5 (BOINC 5.8.13)
1137 / 1423

2. Windows XP SP2 (BOINC 5.8.14)
1407 / 1777

The 5.8.15 release has the same bm's, I just didn't write them down.

Live long and BOINC.

---

Paul.