Multi-Threaded Apps

Message boards : Questions and problems : Multi-Threaded Apps
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
noderaser
Avatar

Send message
Joined: 2 Jan 14
Posts: 276
United States
Message 56098 - Posted: 18 Sep 2014, 4:52:36 UTC

Just wondering why more projects don't use multi-threaded apps... At this point, the only one that I participate in that makes use of multi-threaded apps is BURP. I know there were some questions about full CPU utilization there (i.e., a 4-threaded app vs. 4 single threads) but it would seem more efficient with system memory as the number of cores/threads in CPUs increase.
My Detailed BOINC Stats
ID: 56098 · Report as offensive
Coleslaw
Avatar

Send message
Joined: 23 Feb 12
Posts: 198
United States
Message 56106 - Posted: 18 Sep 2014, 17:30:59 UTC

Because BOINC still doesn't score MT apps fairly if using CreditNew. I have been testing YAFU for a few weeks and points still aren't leveled out properly.

http://yafu.myfirewall.org/yafu/forum_thread.php?id=147

Some apps aren't coded for MT. So, you will have projects that can't modify their code. I agree as far as memory or a few other resources go, it would be nice to have. However, BOINC doesn't seem to be quite ready yet.

Other projects that use MT include:

Milkyway
Moo! (or at least I think it did with GPU's)
Edges (I think)
Yafu
LHC's Test4Theory/vLHC
Renderfarm.fi
GPUGrid (testing)
ID: 56106 · Report as offensive
Ken Putnam

Send message
Joined: 3 Nov 05
Posts: 15
United States
Message 56162 - Posted: 22 Sep 2014, 17:07:18 UTC
Last modified: 22 Sep 2014, 17:08:38 UTC

Is there an app_config operand that will tell a mt app to only use xx% or yy total processors, rather than allowing it to grab EVERY processor on a box


(BURP is another project with a mt app)

(and vLHC uses VBox)
ID: 56162 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56163 - Posted: 22 Sep 2014, 17:31:30 UTC - in response to Message 56162.  

Is there an app_config operand that will tell a mt app to only use xx% or yy total processors, rather than allowing it to grab EVERY processor on a box

Yes, there is.

http://boinc.berkeley.edu/wiki/Client_configuration#Application_configuration

Look at the app_version section (which means that you have to be using v7.2.42 or later).

Set the same figure - number of cores/threads you want the app to use - in both <avg_ncpus> (controls scheduling the other cores) and after <cmdline>--nthreads (controls the behaviour of the MT app itself). That rather assumes that the particular app you're interested in follows the standard rules of behaviour, and I don't think many people have tested it yet.

I'd assume from the use of cmdline that the application behaviour will only change when a new task starts, or at least re-starts.
ID: 56163 · Report as offensive
Ken Putnam

Send message
Joined: 3 Nov 05
Posts: 15
United States
Message 56165 - Posted: 22 Sep 2014, 18:50:31 UTC - in response to Message 56163.  
Last modified: 22 Sep 2014, 19:00:12 UTC


Set the same figure - number of cores/threads you want the app to use - in both <avg_ncpus> (controls scheduling the other cores) and after <cmdline>--nthreads (controls the behaviour of the MT app itself). That rather assumes that the particular app you're interested in follows the standard rules of behaviour, and I don't think many people have tested it yet.

I'd assume from the use of cmdline that the application behaviour will only change when a new task starts, or at least re-starts.



Thanks

Here is my app_config file

<app_config>
<app>
<name>blender</name>
<avg_ncpus> 4</avg_ncpus>
</app_config>
[<app_version>
<app_name>BLENDER</app_name>
[<plan_class>mt</plan_class>]
[<cmdline>--nthreads 4</cmdline>]
</app_version>]


Doesn't appeara to ahve any affect at all

The mt WU still uses all 8 CPUs
ID: 56165 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56166 - Posted: 22 Sep 2014, 18:58:07 UTC - in response to Message 56165.  


Set the same figure - number of cores/threads you want the app to use - in both <avg_ncpus> (controls scheduling the other cores) and after <cmdline>--nthreads (controls the behaviour of the MT app itself). That rather assumes that the particular app you're interested in follows the standard rules of behaviour, and I don't think many people have tested it yet.

I'd assume from the use of cmdline that the application behaviour will only change when a new task starts, or at least re-starts.



Thanks

Here is my app_config file

<app_config>
<app>
<name>blender</name>
<avg_ncpus>4</avg_ncpus>
<cmdline>--nthreads 4</cmdline>
</app_config>

What it did was limit the mt task to 1 CPU, but that WU has still locked out the rest of my apps

Any more thoughts?

You didn't place the settings inside an app_version block which specified which version of which application they referred to. How many CPUs does that computer have?
ID: 56166 · Report as offensive
Ken Putnam

Send message
Joined: 3 Nov 05
Posts: 15
United States
Message 56167 - Posted: 22 Sep 2014, 19:03:01 UTC - in response to Message 56166.  
Last modified: 22 Sep 2014, 19:23:24 UTC

It is a quad core hyper threaded = 8 processors

I realized my mistake and changed the app_config to

<app_config>
<app>
<name>blender</name>
<avg_ncpus> 4</avg_ncpus>
</app_config>
[<app_version>
<app_name>BLENDER</app_name>
[<plan_class>mt</plan_class>]
[<cmdline>--nthreads 4</cmdline>]
</app_version>]


and it is now back to using all 8 processors


Does the app_name have to be the exact name of the executable?

Thanks again for your help


Update

Interestingly enough, tho it is using all 8 processors, they are each only being used at approx 50% ??
ID: 56167 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56168 - Posted: 22 Sep 2014, 19:24:33 UTC - in response to Message 56167.  
Last modified: 22 Sep 2014, 19:26:10 UTC

It is a quad core hyper threaded = 8 processors

I realized my mistake and changed the app_config to

<app_config>
<app>
<name>blender</name>
<avg_ncpus> 4</avg_ncpus>
</app_config>
[<app_version>
<app_name>BLENDER</app_name>
[<plan_class>mt</plan_class>]
[<cmdline>--nthreads 4</cmdline>]
</app_version>]


and it is now back to using all 8 processors


Does the app_name have to be the exact name of the executable?

Thanks again for your help

1) You have to maintain the nested structure of an xml file - inner tags surrounding by middling tags surrounded by outer tags - like a set of russian dolls. </app_config> will be right at the end, to match <app_config> at the beginning, and so on.

2) Square brackets represent optional components. Don't leave the brackets there - put in the option, or take it out entirely.

3) <app_name> isn't the name of an executable file, but the tag for the 'application' used by BOINC to identify the processing group in question. You probably need to be looking at - BUT NOT CHANGING - client_state.xml to find the exact form of all the components in use - yes, the spelling and capitalisation has to be right. You'll rapidly learn that computers are STUPID - they can't work anything out for themselves, they have to be told what to do, in excruciatingly precise detail.

34) If you have 8 cores and want to run 4 threads, you probably want to put in a <max_concurrent> as well, otherwise BOINC will simply run two separate tasks amd you'll be back up to 8 again. See 'STUPID', above.
ID: 56168 · Report as offensive
Ken Putnam

Send message
Joined: 3 Nov 05
Posts: 15
United States
Message 56170 - Posted: 22 Sep 2014, 20:07:59 UTC - in response to Message 56168.  
Last modified: 22 Sep 2014, 20:09:17 UTC

here is the new one

<app_config>
<app>
<name>Blender</name>
<avg_ncpus>4</avg_ncpus>
<max_concurrent>1</max_concurrent>
<app_version>
<app_name>Blender</app_name>
<plan_class>mt</plan_class>
<cmdline>--nthreads 4</cmdline>
</app_version>
</app_config>

Blender (spelled that way in the client_state file) is still using 100% of all 8 processors and displays "(8 CPUs)" in the status field of BOINC Manager

(the anomaly I mentioned before appears to be that there is end of WU processing that does not use all the capacity of the processors that it has grabbed)
ID: 56170 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56171 - Posted: 22 Sep 2014, 20:35:56 UTC - in response to Message 56170.  

here is the new one

<app_config>
<app>
<name>Blender</name>
<avg_ncpus>4</avg_ncpus>
<max_concurrent>1</max_concurrent>
<app_version>
<app_name>Blender</app_name>
<plan_class>mt</plan_class>
<cmdline>--nthreads 4</cmdline>
</app_version>
</app_config>

Blender (spelled that way in the client_state file) is still using 100% of all 8 processors and displays "(8 CPUs)" in the status field of BOINC Manager

(the anomaly I mentioned before appears to be that there is end of WU processing that does not use all the capacity of the processors that it has grabbed)

I don't see a </app> tag to close off the first section, before the beginning of the <app_version> section.

It is easiest to spot if you are absolutely consistent and rigid with your indenting, four spaces each level. And wasn't avg_ncpus in the other section?

<app_config>
<app>
<name>Blender</name>
<max_concurrent>1</max_concurrent>
</app>
<app_version>
<app_name>Blender</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>4</avg_ncpus>
<cmdline>--nthreads 4</cmdline>
</app_version>
</app_config>
ID: 56171 · Report as offensive
Ken Putnam

Send message
Joined: 3 Nov 05
Posts: 15
United States
Message 56172 - Posted: 22 Sep 2014, 20:59:51 UTC - in response to Message 56171.  
Last modified: 22 Sep 2014, 21:10:14 UTC

<app_config>
<app>
<name>Blender</name>
<avg_ncpus>4</avg_ncpus>
<max_concurrent>1</max_concurrent>
</app>
<app_version>
<app_name>Blender</app_name>
<plan_class>mt</plan_class>
<cmdline>--nthreads 4</cmdline>
</app_version>
</app_config>

Stupid board strips leading spaces - I tried to formatting it myself, but apparently just hosed things up

Thanks again for all your assistance


Still 100% of all processors <sigh>
ID: 56172 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56174 - Posted: 22 Sep 2014, 21:35:05 UTC - in response to Message 56172.  

<app_config>
   <app>
       <name>Blender</name>
       <avg_ncpus>4</avg_ncpus> 
       <max_concurrent>1</max_concurrent>
   </app>
   <app_version>
       <app_name>Blender</app_name>
       <plan_class>mt</plan_class>
       <cmdline>--nthreads 4</cmdline>
   </app_version>
</app_config>

You're supposed to be able to display them with [ pre ], but it's not working. Jord was wrestling with that a few days ago...
ID: 56174 · Report as offensive
Coleslaw
Avatar

Send message
Joined: 23 Feb 12
Posts: 198
United States
Message 56175 - Posted: 22 Sep 2014, 22:01:16 UTC - in response to Message 56162.  

Is there an app_config operand that will tell a mt app to only use xx% or yy total processors, rather than allowing it to grab EVERY processor on a box


(BURP is another project with a mt app)

(and vLHC uses VBox)


Ken, I am aware that BURP is also a project with a MT app. The OP already mentioned so I didn't feel it needed repeated. And yes vLHC uses VBox but that doesn't change that it is a multi-thread app.

Also, your quad only has 1 CPU. You are saying CPU when you mean cores or in this case potentially threads since you mentioned hyper threading. So, you have 1 cpu that has 4 cores and a total of 8 threads. BOINC however, will report them as CPU's which isn't really accurate.
ID: 56175 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 56176 - Posted: 22 Sep 2014, 22:05:17 UTC - in response to Message 56174.  

Jord was wrestling with that a few days ago...

Yeah, but that's because I have the project administrator title. I can do HTML on these forums, if I want to. I cannot use the 'normal' smaller than and greater than signs but instead have to reside to their HTML counterparts if I want to use them around containers.

But I believe that some of that code has now been completely wrecked, so all bets are off. Anyone doing things with [pre/] and [code/] containers is on his own. ;-)
ID: 56176 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56178 - Posted: 22 Sep 2014, 22:43:51 UTC - in response to Message 56172.  

Still 100% of all processors <sigh>

Well, I said in my first reply that I hadn't seen anybody really testing the new config feature yet. And new code from BOINC has a habit of not working first time. Sigh. OK, I'll find myself an MT project one of these fine months and see what I can make it do.

FWIW, I did try BURP years ago, going down the anonymous platform route, and it responded properly to the --nthreads command - but I detached again because I wanted the disk space back. So I'll have to start from scratch. Good thing I'm on unlimited downloads.
ID: 56178 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56214 - Posted: 24 Sep 2014, 14:07:59 UTC - in response to Message 56176.  

Jord was wrestling with that a few days ago...

Yeah, but that's because I have the project administrator title. I can do HTML on these forums, if I want to. I cannot use the 'normal' smaller than and greater than signs but instead have to reside to their HTML counterparts if I want to use them around containers.

But I believe that some of that code has now been completely wrecked, so all bets are off. Anyone doing things with [pre/] and [code/] containers is on his own. ;-)

Talking of BOINC code not working first time, the broken [pre/] and [code/] containers have now migrated to SETI and (I presume) SETI Beta.
ID: 56214 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 56248 - Posted: 25 Sep 2014, 19:27:48 UTC - in response to Message 56174.  

<app_config>
   <app>
       <name>Blender</name>
       <avg_ncpus>4</avg_ncpus> 
       <max_concurrent>1</max_concurrent>
   </app>
   <app_version>
       <app_name>Blender</app_name>
       <plan_class>mt</plan_class>
       <cmdline>--nthreads 4</cmdline>
   </app_version>
</app_config>

Yay! [ pre ] shows spaces and XML tags properly now!
ID: 56248 · Report as offensive
Grandpa

Send message
Joined: 20 Jun 14
Posts: 11
United States
Message 56279 - Posted: 27 Sep 2014, 2:34:43 UTC - in response to Message 56248.  
Last modified: 27 Sep 2014, 2:36:13 UTC

You can limit the number of threads in the cc_config by adding this <ncpus>48</ncpus> it has worked for me, but I have never tried limiting the number of thread through app_config.

<cc_config>
<options>
<ncpus>48</ncpus>
</options>
</cc_config>
ID: 56279 · Report as offensive
SuperSluether

Send message
Joined: 6 Jul 14
Posts: 94
United States
Message 56284 - Posted: 27 Sep 2014, 16:39:33 UTC - in response to Message 56098.  

Maybe I'm misunderstanding, but hopefully I'm not. Multi-threaded apps (tasks running on more than 1 core) use OpenCL to do so. This basically helps the task complete faster by using parallel processing instead of serial.

However, some projects can't use this, or there's no need becuase certain tasks won't benefit from it. Another benefit of running single-thread tasks is that you can run more than 1 task, and run tasks from more than 1 project at the same time.

Like I said, hopefully I understood this correctly. It's interesting to note that from Collatz Conjecture, single-core and multi-core tasks take the same amount of time to do about the same amount of processing, so for that project the OpenCL doesn't really have a benefit.
ID: 56284 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 56287 - Posted: 27 Sep 2014, 16:50:49 UTC - in response to Message 56284.  

Maybe I'm misunderstanding, but hopefully I'm not. Multi-threaded apps (tasks running on more than 1 core) use OpenCL to do so.

Not necessarily. Before the advent of OpenCL (on CPUs), projects could already code multi-threaded applications in C, C++ and other languages.
ID: 56287 · Report as offensive
1 · 2 · Next

Message boards : Questions and problems : Multi-Threaded Apps

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.