Running only one process but let it use multiple CPUs

Message boards : API : Running only one process but let it use multiple CPUs
Message board moderation

To post messages, you must log in.

AuthorMessage
jkalmar

Send message
Joined: 23 Jan 14
Posts: 3
Slovakia
Message 52108 - Posted: 23 Jan 2014, 16:34:36 UTC
Last modified: 23 Jan 2014, 16:37:27 UTC

Hello I'm new to this forum.

We have a multi-threaded app, we can limit the max threads it will use in the app itself.

Now, the user can set in boinc-manager to use only a certain number of CPUs(in %), but that will run also a certain number of processes (instances of the app), is there a way to always run only one instance of app (one process) and give it the number of threads it can use according to the user settings in boinc-manager?

I have already read the wiki, found this: http://boinc.berkeley.edu/trac/wiki/AppMultiThread but that didn't helped me.
ID: 52108 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 305
Germany
Message 52120 - Posted: 24 Jan 2014, 11:14:03 UTC

Hello,

number of CPU equals the number of processes in this special case. At the moment you have to specify for your application how many threads to start (not dynamic) and the client will take this into account when scheduling other tasks. So if your application uses 2 Threads, one task will block 2 CPUs and will only get started when 2 CPUs are available (according to the settings and the workload and priorities of other projects/tasks).

This dynamic mode you want (Client telling the app how much CPUs are free) is not implemented at the moment and seems to be difficult with the current client scheduler. For more information you should contact the boinc_dev mailinglist with your request.
ID: 52120 · Report as offensive
Richard Haselgrove
Volunteer moderator
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 2765
United Kingdom
Message 52130 - Posted: 24 Jan 2014, 20:19:56 UTC - in response to Message 52108.  

Hello I'm new to this forum.

We have a multi-threaded app, we can limit the max threads it will use in the app itself.

Now, the user can set in boinc-manager to use only a certain number of CPUs(in %), but that will run also a certain number of processes (instances of the app), is there a way to always run only one instance of app (one process) and give it the number of threads it can use according to the user settings in boinc-manager?

I have already read the wiki, found this: http://boinc.berkeley.edu/trac/wiki/AppMultiThread but that didn't helped me.

The links at the bottom of that page take you to a discussion of multi-threaded applications, and the default "mt" plan_class.

That seems to go part way towards what you want. 'mt' schedules a single application instance, and configures it to use every available CPU core.

I'm familiar with running the OpenMP type of application (mentioned in the Wiki) under Windows. In that particular case, the configuration is in two parts: a directive to BOINC (ncpus) telling the client how many cores the application is going to need (all of them, by default, but it's made explicit); and a directive to the application (cmdline --nthreads) telling it how many threads to start (likewise, by default explicitly enumerating the number of available cores).

Given those two degrees of configuration, it would be possible to copy and adapt the default 'mt' plan_class code to specify some function other than 100% of available cores as the number of threads to be scheduled and used (keep them the same). You might deploy the application for 75% of available cores, for example, or for (n-2) cores.

Extending the concept beyond that, you could envisage adding a project custom preference to the web page, allowing users to specify the number of cores they are willing to make available to your application. Then, that number could be passed via a custom plan_class based on 'mt' with the work allocated to the user. Again, I'd recommend that you pass the user's preferred number both as ncpus to control scheduling, and as --nthreads to control the application - and again, keep them the same: that should be possible without too much extra work. But I think you'd have to ask users to configure it via the website, rather than directly in BOINC Manager.
ID: 52130 · Report as offensive
jkalmar

Send message
Joined: 23 Jan 14
Posts: 3
Slovakia
Message 52147 - Posted: 26 Jan 2014, 21:05:29 UTC

Thanks for help.

So there is no way to do it dynamically, using the current api.

I looked at the wiki and found that in the project config you can specifi that boinc will run only single instacne of the app, now we just need to query the boinc client config file and get the <max_ncpus_pct> VALUE </max_ncpus_pct> from it.

There are only few problems:
1. how can I get in my app the file location? in my linux mint it is located in /etc/boinc-client/global_prefs_override.xml but I have no idea where it is stored in windows nor if it is in the same path on other linux distros or mac.

2. you will have to wait for the current processed wu to be finished and then apply the new settings (if the user change his settings).

The second problem is a minor one - it can be solved in our app, but the first one is quite serious. So is there some way to get the config from client or at least the file location?

Thanks
ID: 52147 · Report as offensive
Richard Haselgrove
Volunteer moderator
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 2765
United Kingdom
Message 52148 - Posted: 26 Jan 2014, 21:44:19 UTC - in response to Message 52147.  

I think you'll find that all of that is handled by the MT plan_class mechanism.

From my experience (purely as a Windows user), the necessary information is parsed by the client from the user's preferences, and passed to the server as part of the work request.

Thus, if I have an 8 core (8 thread) CPU, and I have set my project preference to "On multiprocessors, use at most 75% of the processors", then my computer will ask for, and be allocated, work as if it is a 6-core computer.

If the project is issuing single-threaded work, I'll get a collection of tasks, and the client will schedule them 6 at a time, to use my preferred number of cores.

If the project is issuing MT work, the tasks I receive will already be configured to use my preferred number of cores - 6, in this case. But that will take up as much of my computer as I have said I'm prepared to make available to BOINC: so no other project's task will be able to share the machine.

The problem I was mentioning before only arises, and programming is only needed, if you as a project don't want to monopolise all the resources I offer to BOINC, whether that is 100% of my physical hardware or some lower proportion expressed through my preferences.
ID: 52148 · Report as offensive

Message boards : API : Running only one process but let it use multiple CPUs

Copyright © 2018 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.