Suggestion - add GPU task's CPU usage into calculations

Message boards : BOINC client : Suggestion - add GPU task's CPU usage into calculations
Message board moderation

To post messages, you must log in.

AuthorMessage
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 68791 - Posted: 4 Apr 2016, 22:04:17 UTC - in response to Message 68790.  
Last modified: 4 Apr 2016, 22:05:45 UTC

Since each GPGPU using project has a different CPU 'support' need (to keep the GPGPU task running) and the variety and combinations of GPGPU capability and CPU are endless, 'you' can set the needs at project level in the cc_config.xml + app_config.xml file to maximize/optimize utilization.

<app_config>
   [<app>
      <name>Application_Name</name>
      <max_concurrent>1</max_concurrent>
      [<fraction_done_exact/>]
      <gpu_versions>
          <gpu_usage>.5</gpu_usage>
          <cpu_usage>.4</cpu_usage>
      </gpu_versions>
    </app>]
   ...
   [<app_version>
       <app_name>Application_Name</app_name>
       [<plan_class>mt</plan_class>]
       [<avg_ncpus>x</avg_ncpus>]
       [<ngpus>x</ngpus>]
       [<cmdline>--nthreads 7</cmdline>]
   </app_version>]
   ...
   [<project_max_concurrent>N</project_max_concurrent>]
</app_config>

Manual at: http://boinc.berkeley.edu/wiki/Client_configuration
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 68791 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 68798 - Posted: 5 Apr 2016, 11:08:45 UTC - in response to Message 68797.  

[ ... ] is the convention used in computer documentation to indicate an optional element.

If you need to use it, remove the square brackets and supply an appropriate value. If you don't need to use it, remove the brackets and all contents between the opening bracket and the matching closing bracket.
ID: 68798 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 68802 - Posted: 5 Apr 2016, 13:23:46 UTC - in response to Message 68800.  

Thanks, now I'm getting somewhere. But it needs the application name. I thought I could just tell it to give Einstein a certain amount of CPU per GPU task, but surely Einstein is constantly using different applications, so I'd have to manually check up and edit this file all the time.

You can put in multiple entries, one for each application you choose run run. And then just leave them alone.

Can I put "all" or "*" or something in the application name?

No.
ID: 68802 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 68805 - Posted: 5 Apr 2016, 14:54:50 UTC - in response to Message 68803.  

BOINC already has such a calculation. It will over-commit the CPU by, strictly, "less than one CPU-corer". That, in most cases, allows applications that run mostly on GPUs to coexist and make progress alongside applications that run mostly on CPUs.

It's a fallacy that a scientific application ever runs exclusively on one CPU, to the exclusion of everything else. Last time I bothered to count, there were something like 100 programs running across the four CPU cores of the Windows 7 machine I was looking at - thanks to the efficient implementation of time-division multiplexing in modern operating systems.

It is true that BOINC's automatic algorithm for working out how many fractions of a CPU a GPU application is going to need for its overheads on any particular CPU/GPU hardware combination is pretty rudimentary - but it tends to err on the side of generosity: it sets a higher 'fractional CPU' number than most GPU apps need. That, in turn, brings closer the point at which all the fractions add up to 1.0 or more, and at that point a CPU task is removed from the scheduling, just as you're requesting.

And please remember that the scientists who develop the applications we run are primarily interested in finding gravitational waves, binary pulsars, or whatever. You may feel that you don't have enough time to test all combinations on just one machine - a computer you presumably know in some detail: the astrophysicists at Einstein certainly don't have time to test all the combinations which volunteers have brought to their project.
ID: 68805 · Report as offensive
Profile Agentb
Avatar

Send message
Joined: 30 May 15
Posts: 265
United Kingdom
Message 68848 - Posted: 7 Apr 2016, 0:10:22 UTC - in response to Message 68814.  

... so it should be simple


Yes ideally, but it's not simple. Distributed computing is not simple and GPU based computing is very hard. If it were easy, we would see it used in main stream apps. It is really difficult to write GPU based applications and for operating systems to control them in a nice way.

Even measuring the "speed" of a GPU will give you vastly different numbers depending on "what test". The number will vary depending on the number and mixture of tests and the how much data they have to move, the data pathways and their speeds.

Measuring FLOPS of an normal application is guesswork, and the time the application runs depends on many things other than the number of floating point operations it has to execute. Some apps are memory bound etc. GPU tasks have to find PCIE-bandwidth to page out huge amounts of memory etc.

boinc has to make guesses about what works "ok", in a middle ground sort of way, and the guesses are not always going to be perfect due to the vast variety in apps / OS and hardware, especially when you want to push to the limit.

If boinc is not running perfectly (whatever that means), there are many parameters we can tune to get more out. CPU percentages globally or adjust by application etc etc. My preference is use the GPU as best you can, and leave the CPU mostly idle, so not being busy when the GPU needs it.

Others may want to go CPU first and not use the GPU so much as "GPU tasks are taking over my system". Low end graphics cards are not pleasant to use when busy crunching.

If an app says 0.5 and i see it running better - on my system at 0.25 or 0.75 i'll change it, and i'll leave a core per card.
ID: 68848 · Report as offensive

Message boards : BOINC client : Suggestion - add GPU task's CPU usage into calculations

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.