Big ask - auto throttling of CPU tasks when using GPUs?

Message boards : GPUs : Big ask - auto throttling of CPU tasks when using GPUs?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4862
United Kingdom
Message 96235 - Posted: 1 Mar 2020, 13:01:11 UTC - in response to Message 96234.  

Yes. Use an app_config.xml file and 'declare' that the GPU tasks require 100% of a CPU (core) in support. Doesn't change the actual GPU application behaviour, but does change BOINC's scheduling.
ID: 96235 · Report as offensive
Nick Name

Send message
Joined: 14 Aug 19
Posts: 55
United States
Message 96246 - Posted: 1 Mar 2020, 20:30:11 UTC - in response to Message 96244.  

There's no way to "automatically" do what you want or do any dynamic load balancing, at least within BOINC itself. The easiest thing to do is lower your CPU usage limit. BOINC will typically over-commit the CPU when running GPU work, e.g. if BOINC is set to use four CPU threads and you have two GPUs BOINC will run six jobs in total. I take this into account and set CPU use so the number of jobs doesn't exceed the CPU thread count.
Team USA forum
Follow us on Twitter
Help us #crunchforcures!
ID: 96246 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4862
United Kingdom
Message 96249 - Posted: 1 Mar 2020, 20:58:21 UTC - in response to Message 96244.  

Yes I'm already doing that. But the problem is different tasks require different amounts of CPU. If you're running several projects it can become quite complicated. And other things use the CPU, like the user using the computer. It's very variable and would be better monitored 24/7 by the computer itself. It must be fairly easy for Boinc to monitor the CPU usage, and if it's hitting 100%, lower the number of CPU tasks running. Then no GPU tasks will ever get throttled, and the user will never experience a slowdown of the interface.
I think you're asking for something like #2949
ID: 96249 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 714
United States
Message 96256 - Posted: 2 Mar 2020, 3:48:56 UTC

Indeed. Gpugrid uses significantly higher CPU usage than the collatz, even though the numbers aren't fully updated to reflect this.
ID: 96256 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15114
Netherlands
Message 96509 - Posted: 8 Mar 2020, 17:36:53 UTC - in response to Message 96507.  

But BOINC doesn't set run priority of applications, as it doesn't run any work. BOINC is just the managing software. It's the project's science applications that do the work and it's the project that sets the run priority of the applications when they compile said applications. BOINC documentation does advise on what to set it to but it's still up to the project to follow that advice.
ID: 96509 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 714
United States
Message 96510 - Posted: 8 Mar 2020, 17:45:25 UTC - in response to Message 96507.  

I was wondering, why doesn't Boinc just set CPU tasks as lowest priority in Windows, and GPU tasks as a higher priority, then the first should never get in the way of the second.

So I checked. It already does this, I guess Windows is rubbish. How can an OS get such a basic thing wrong?


I don't know about Windows, but I do know in Linux this is the case.
I'm running 5 tasks on a dual core + HT CPU (Pentium Gold G6500). 2 of them are CPU, 2 are Nvidia GPU, and 1 is the Intel IGP.
Boinc seems to focus on getting all GPU (including the weak Intel IGP) active.
That being said, the IGP is about as powerful as the CPU (200Gflops), so losing 1 thread to the IGP is a win anyway (lose <50Gflops on CPU, gain 200GFlops).
ID: 96510 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4862
United Kingdom
Message 96511 - Posted: 8 Mar 2020, 18:35:27 UTC - in response to Message 96509.  

But BOINC doesn't set run priority of applications, as it doesn't run any work. BOINC is just the managing software. It's the project's science applications that do the work and it's the project that sets the run priority of the applications when they compile said applications. BOINC documentation does advise on what to set it to but it's still up to the project to follow that advice.
'Priority' is a difficult word, because it's used in so many settings to mean so many things. But if you mean the thread priority of the CPU (part of a) project science app - yes, it does.

It's always set full-CPU apps to run at idle priority, so as not to get in the way of the foreground apps the user is using. And when GPUs came along, their apps were set to 'below normal'. It's done when the BOINC client is starting the science app, as part of its management role.
ID: 96511 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4862
United Kingdom
Message 96516 - Posted: 8 Mar 2020, 19:45:00 UTC - in response to Message 96514.  

'Utilisation' is not the same thing as 'priority'. A GPU, in particular, will only be 100% utilised if the problem is amenable to parallelisation, and the programmer is clever enough to program it that way.
ID: 96516 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15114
Netherlands
Message 96520 - Posted: 8 Mar 2020, 20:53:24 UTC - in response to Message 96511.  

It's done when the BOINC client is starting the science app, as part of its management role.
It's done when you compile the program using the BOINC API:
https://boinc.berkeley.edu/trac/wiki/BasicApi, here you set if the app is single- or multi-threaded, which priority it runs at, whether it checkpoints, whether it uses a wrapper, etc.
ID: 96520 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4862
United Kingdom
Message 96523 - Posted: 8 Mar 2020, 21:11:48 UTC - in response to Message 96520.  

It's done when the BOINC client is starting the science app, as part of its management role.
It's done when you compile the program using the BOINC API:
https://boinc.berkeley.edu/trac/wiki/BasicApi, here you set if the app is single- or multi-threaded, which priority it runs at, whether it checkpoints, whether it uses a wrapper, etc.
I prefer to look at the actual code:

https://github.com/BOINC/boinc/blob/master/client/app_start.cpp#L570

// Start a task in a slot directory.
// This includes setting up soft links,
// passing preferences, and starting the process
...
// run it at above idle priority if it
// - uses coprocs
// - uses less than one CPU
// - is a wrapper
Not the first time the code has diverged from the documentation, over time.
ID: 96523 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4862
United Kingdom
Message 96524 - Posted: 8 Mar 2020, 21:16:55 UTC - in response to Message 96517.  
Last modified: 8 Mar 2020, 21:30:18 UTC

But what I was observing was if the CPU is too busy with unrelated tasks (eg CPU WUs), then the GPU slowed down. This suggests to me that the CPU part of that GPU WU was not being given enough CPU time. But it should have been, as its priority in the OS was higher than the CPU WUs.
Not necessarily. It might be that the CPU had to refetch a lot of data that wasn't in local cache memory, at every 'context switch' (look up that one up too).
ID: 96524 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15114
Netherlands
Message 96526 - Posted: 8 Mar 2020, 21:36:47 UTC - in response to Message 96523.  

Not the first time the code has diverged from the documentation, over time.
Gotta wonder though which one it is now. I'm not going to ask.
ID: 96526 · Report as offensive
1 · 2 · Next

Message boards : GPUs : Big ask - auto throttling of CPU tasks when using GPUs?

Copyright © 2022 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.