"--fetch_minimal_work" really fetches 1 work unit per project .. expected?

Message boards : Questions and problems : "--fetch_minimal_work" really fetches 1 work unit per project .. expected?
Message board moderation

To post messages, you must log in.

AuthorMessage
Bluefin Tuna

Send message
Joined: 9 Jun 13
Posts: 3
Luxembourg
Message 49555 - Posted: 9 Jun 2013, 20:10:49 UTC

Hi,

I have a machine here with 8 CPUs or so running boinc 7.0.65

I noticed that when I use the option "--fetch_minimal_work" for the boinc client, then 1 work unit per project is fetched. As I have two projects, this means two CPUs are being used.

Is this expected behaviour? I would have thought I work unit per CPU would be fetched so that all CPUs would have something to do.

ID: 49555 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15483
Netherlands
Message 49556 - Posted: 9 Jun 2013, 21:59:23 UTC - in response to Message 49555.  

http://boinc.berkeley.edu/wiki/Client_configuration writes:
--fetch_minimal_work
Fetch only 1 job per device (CPU, GPU).


So yes, works at it should.
Before you ask, you have only 1 CPU. It may have 8 cores, but it's only one CPU, == 1 device.
ID: 49556 · Report as offensive
Paul Schauble

Send message
Joined: 29 Aug 05
Posts: 68
Message 49623 - Posted: 15 Jun 2013, 21:25:13 UTC - in response to Message 49556.  

That's not a very useful definition of "device" with respect to CPU cores. Everything else in BOINC schedules by core, not by CPU.

++PLS
ID: 49623 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15483
Netherlands
Message 49624 - Posted: 15 Jun 2013, 22:58:27 UTC - in response to Message 49623.  
Last modified: 15 Jun 2013, 23:00:24 UTC

Do know that the --fetch_minimal_work option is one very good one for debugging applications. Especially when used in combination with --exit_after_finish
The --fetch_minimal_work option has not much to do with how BOINC schedules work. When it sees this option, it'll just ask 1 task per computational piece of hardware, or device.

If you want to cache 1 task per core + amount of GPUs, set "minimum work" and "additional work" to zero. Then BOINC will always only ask enough work to fill the hardware, maybe one or two extra as cache.

Now, not 'everything else in BOINC schedules by core'.
All GPUs are detected 'by device', and shown as device0, device1, device2 etc.
GPUs have hundreds of computation cores. BOINC only schedules by GPU, not by the individual computation cores in the GPU.

So equally, one CPU is a device.
When you dismantle your PC and take the CPU out of its socket, you hold one piece of hardware, one device, not 8.
When you are a lucky person and you have 4 Xeon CPUs on a motherboard, you have 4 devices.
When you're an ultra lucky person with 4 Xeon CPUs and 2 GPUs, you have 6 devices.
It's that simple.
ID: 49624 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 49627 - Posted: 16 Jun 2013, 6:59:05 UTC - in response to Message 49624.  

Just wonder, if with a 4 Xeon Mobo, you get 4 with minimal fetch, but think your 'device' theory here falls over [And then if it were 4 would BOINC know to execute one on each CPU?... doubt it... the OS is in control of that I'd say]. Anyone here with multiple CPU's to give it a test... that is, if making an argumentative point is your thing ;>)
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 49627 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15483
Netherlands
Message 49628 - Posted: 16 Jun 2013, 7:14:39 UTC - in response to Message 49627.  
Last modified: 16 Jun 2013, 7:50:12 UTC

Just wonder, if with a 4 Xeon Mobo, you get 4 with minimal fetch, but think your 'device' theory here falls over

BOINC will see 4 separate CPUs, not one CPU with 4 cores.
I am not saying that BOINC will see 16 CPUs if each Xeon is a quad core.


ID: 49628 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 49629 - Posted: 16 Jun 2013, 9:04:25 UTC - in response to Message 49628.  

This is what my log says with '7.1.17':

15/06/2013 12:26:32 | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz [Family 6 Model 42 Stepping 7]

8 'processors', 1 CPU. I'd imagine a 4 quad Xeon lists out 4 times 4 processors on 4 CPUs. How 'minimum fetch' responds to having these 4 'devices' in them, pass... an owner would be able to give a log snip of the CPU reporting portion and a fetch request. We discussed this over a year ago, if not 2, when we first discovered the min.fetch really only pulled in 1 task. Still looking for that one command, with no work on board that occupies all processors, completes them and exits BOINC. Not been able to work it out with what we have, but that's just me.

Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 49629 · Report as offensive
Bluefin Tuna

Send message
Joined: 9 Jun 13
Posts: 3
Luxembourg
Message 49630 - Posted: 16 Jun 2013, 14:22:23 UTC - in response to Message 49629.  

> It may have 8 cores, but it's only one CPU, == 1 device.

Ok, thanks.

This is a special definition though :-D

A better nomenclature would be:

"CPU SOCKET" (Device) -< "CPU CORE" -< "CPU HYPERTHREADING SIBLING"

Btw, my machine is actually an Amazon EC2 resource, so I have no idea about the physical structure underneath the EC2 Virtual Machine... :-)

For those interested:

https://www.ibm.com/developerworks/community/blogs/brian/entry/linux_show_the_number_of_cpu_cores_on_your_system17?lang=en

Amazon EC2 seems to give one "hyperthread" as a "core"... not quite sure.

Ok, let's see, on this amazon instance, we run:

cat /proc/cpuinfo | egrep "processor|model name|physical id|siblings|core id|cpu cores" | sed 's/processor/\nprocessor/g'


This yields the following:

processor       : 0
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 1
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 2
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 3
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 4
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 5
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 6
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1

processor       : 7
model name      : Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 1


An E5506 has 4 cores, no "hyperthreads", so here with "8 processors" one would have "2 BOINC devices".

However, it seems the VM advertises the same "physical id" (socket #) for each core. It also gives each core the same "core id" on that fanciful socket :-(
ID: 49630 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15483
Netherlands
Message 49659 - Posted: 19 Jun 2013, 16:56:22 UTC

OK, I was wrong. The --fetch_minimal_work algorithm asks for work for all devices once (all CPUs, all cores, all GPUs). But then it won't ask for work anymore afterwards. So you run one task per core, per GPU and that's it.

It also seems to only ask for work from the primary project, not from any backup projects. It also won't ask for work piggy-backed upon a cache. Meaning that if you had work in cache and you enable <fetch_minimal_work/>, that you'll end up with an empty cache when all work already in cache has been crunched.

Only on a restart of the client will it ask for work, get 1 task for each piece of hardware, then cease again.
ID: 49659 · Report as offensive
Bluefin Tuna

Send message
Joined: 9 Jun 13
Posts: 3
Luxembourg
Message 49673 - Posted: 21 Jun 2013, 12:44:49 UTC

There *should* a manpage for BOINC.
ID: 49673 · Report as offensive

Message boards : Questions and problems : "--fetch_minimal_work" really fetches 1 work unit per project .. expected?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.