Posts by Seriema

1) Message boards : Projects : Is this an ok way to benchmark Azure VMs? (Message 100227)
Posted 6 Aug 2020 by Seriema
Post:
@ProDigit That's pretty scary! The reason for me to do this is because I tend to get free credits on Azure.

Here are the results for what it's worth. They're all sorted by "Result/Price". The "Cost 48h" is my actual bill and 1 SEK is about $0.12.

Using free credits
One can get up to 150 USD/month, which only affords a few simple VM types as seen by the rightmost column. A bunch of "B1s" seems like the most cost effective way?

| VM                 | vCPU | Memory (GiB) | Total Run Time (y:d:h:m:s) | Points Generated | Results Returned | Cost 48h (SEK) | Result/Price | Price/Month (USD) |
|--------------------|------|--------------|:--------------------------:|:----------------:|:----------------:|----------------|--------------|-------------------|
| B1ls               |    1 |          0.5 |       -:---:--:--:--       |         0        |         0        |           2.05 |            - |                 4 |
| B1s                |    1 |            1 |       0:001:10:03:29       |       4,204      |         2        |           4.03 | 0.4962779156 |                 9 |
| B2s                |    2 |            4 |       0:002:13:35:10       |      10,231      |         7        |          16.84 | 0.4156769596 |                36 |
| B2ms               |    2 |            8 |       0:003:00:16:29       |      10,994      |        14        |          34.34 | 0.4076878276 |                72 |
| B4ms               |    4 |           16 |       0:007:12:44:33       |      16,711      |        23        |          68.02 | 0.3381358424 |               144 |


Paying yourself
Spot VM's are dirt cheap, but subject to availability and can't use free credits. The GPU enabled "NV4as v4" were not worth it, and Linux easily beats Windows (unsurprisingly, but one guide recommended it for the GPU drivers).

| VM                 | vCPU | Memory (GiB) | Total Run Time (y:d:h:m:s) | Points Generated | Results Returned | Cost 48h (SEK) | Result/Price (SEK) |
|--------------------|------|--------------|:--------------------------:|:----------------:|:----------------:|----------------|--------------|
| F4s_v2             |    4 |            8 |       0:007:21:57:31       |      54,881      |        96        |          14.11 |   6.80368533 |
| NV4as_v4           |    4 |           14 |       0:007:17:43:51       |      57,176      |        103       |          19.64 |  5.244399185 |
| E4as_v4            |    4 |           32 |       0:007:04:48:30       |      54,405      |        97        |          20.24 |  4.792490119 |
| NV4as_v4 (Windows) |    4 |           14 |       0:006:22:32:31       |      39,378      |        70        |          17.06 |  4.103165299 |
2) Message boards : Projects : Is this an ok way to benchmark Azure VMs? (Message 100179)
Posted 2 Aug 2020 by Seriema
Post:
Hi,

I've set up a few Azure VM's and I'm trying to figure out what would give "most bang for the buck". I've never been deep in the benchmarking world so after some time scouring the Internet I came out empty (too many "it depends") and decided to do this:


  1. Use BOINC
  2. Connected to my World Community Grid (WCG) with an Account Key so they all run the same project (COVID-19).
  3. Let each VM run for 48 hours from "Date Installed & Registered", then shut it down
  4. Whatever's reported to WCG "Device Statistics History" during that time is what I'll use
    * This might mean some VM was seconds away from returning a new batch of results
  5. Wait half a day or so for WCG results ("Last Updated" message always says something around "6 hour(s) ago")
  6. Wait at least a day for Azure costs to be calculated
  7. Post results on this forum, see example table below



Example results table:

| VM   | vCPU | Memory | GPU? | Total Run Time | Points Generated | Results Returned | Cost Listed USD/month | Cost relative | Cost Actual 48h | Cost used Spot price? |
|------|------|--------|------|----------------|------------------|------------------|-----------------------|---------------|-----------------|-----------------------|
| B2ms | 2    | 8gb    | no   | 0:001:12:17:19 | 6,306            | 7                | 60.74                 | 2             | X               | Yes                   |
| B4ms | 4    | 16gb   | no   | 0:003:11:14:29 | 9,560            | 11               | 121.18                | 3.99          | Y               | No                    |
| B2s  | 2    | 4gb    | no   | 0:000:06:40:46 | 1,162            | 1                | 30.37                 | 1             | Z               | No                    |


Any suggestions on how to improve my approach? Anything I should change, add, or remove? Is there a good console command for getting some useful stats worth sharing? I'll be shutting off the first batch of VM's tonight, and the rest tomorrow.




Copyright © 2021 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.