Cant figure out how to get more than 50 threads running

Message boards : Questions and problems : Cant figure out how to get more than 50 threads running
Message board moderation

To post messages, you must log in.

AuthorMessage
usao
Avatar

Send message
Joined: 15 Apr 15
Posts: 50
United States
Message 61767 - Posted: 21 Apr 2015, 13:49:38 UTC

Ive looked through the project config XML files and can't find any reason that Boinc would limit itself to 50 threads.
Does anyone know if there is somthing in the boinc_client program itself which limits to 50 threads, or perhaps knows where to look for such a limit?
Can it be raised beyond 50 somehow, if so, please let me know where to look.
Thanks
ID: 61767 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15482
Netherlands
Message 61768 - Posted: 21 Apr 2015, 13:57:29 UTC - in response to Message 61767.  
Last modified: 21 Apr 2015, 13:58:14 UTC

And again, before we have to play a further 20 questions to get the least bit of information out of you, please read when requesting help on these forums.

Show the messages.
Post what you set in your configuration XML files.
Show what BOINC says when it contacts the project, and the scheduler answers.
Play around with the client configuration debug flags specifically the <sched_op_debug> and maybe a round of <work_fetch_debug>

You're only shooting yourself in the foot by ignoring the requests for information.
ID: 61768 · Report as offensive
usao
Avatar

Send message
Joined: 15 Apr 15
Posts: 50
United States
Message 61770 - Posted: 21 Apr 2015, 14:05:11 UTC - in response to Message 61768.  
Last modified: 21 Apr 2015, 14:06:19 UTC

Sorry, im use to working in a rapid prototype environment at work. We often just throw Q&A around quickly and I forgot that others don't work this way by default.

OS: CentOS release 6.6 x64
CPU: 8x X6550 (128 cores total)
RAM: 512Gb
GPU: None

> boinc_client --version
7.2.33 x86_64-pc-linux-gnu
boinc-client-7.2.33-3.git1994cc8.el6.x86_64

HostID: 616707
Project: MilkyWay@Home (Skipping N-Body simulation)

Not running any manager or services, just running boinc_client as a stand alone process on the command line. Box is dedicated for Boinc.
ID: 61770 · Report as offensive
usao
Avatar

Send message
Joined: 15 Apr 15
Posts: 50
United States
Message 61771 - Posted: 21 Apr 2015, 14:10:00 UTC - in response to Message 61770.  

Im getting from the second instance of boinc_client:

21-Apr-2015 06:43:40 [---] GUI RPC bind to port 31416 failed: 98
21-Apr-2015 06:44:10 gstate.init() failed
Error Code: -180


Looks like it's a possibly a port conflict with the first instance???
I grepped around for the RPC details in the existing XML files, but don't see any references.
ID: 61771 · Report as offensive
BobCat13

Send message
Joined: 6 Dec 06
Posts: 118
United States
Message 61772 - Posted: 21 Apr 2015, 14:24:16 UTC - in response to Message 61771.  

Im getting from the second instance of boinc_client:

21-Apr-2015 06:43:40 [---] GUI RPC bind to port 31416 failed: 98
21-Apr-2015 06:44:10 gstate.init() failed
Error Code: -180


Looks like it's a possibly a port conflict with the first instance???
I grepped around for the RPC details in the existing XML files, but don't see any references.

To get the second client to run properly, I believe the command line should be something like this:

.boinc --daemon --allow_multiple_clients --gui_rpc_port 31417 --dir /home/rwatkins/.hp6b

That may not be 100% correct as it has been a long time since I tested using multiple clients.

Also, I know MilkyWay has a limit of 3 tasks per core, but does anyone know if they also have a limit of 50 tasks per client instance?
ID: 61772 · Report as offensive
usao
Avatar

Send message
Joined: 15 Apr 15
Posts: 50
United States
Message 61773 - Posted: 21 Apr 2015, 14:30:01 UTC - in response to Message 61772.  

Changing the RPC port did the trick aparently.
I now have 98 processes running, up from 50.

I used the following command:

boinc_client --allow_multiple_clients --no_gpus --gui_rpc_port 31417 --attach_project ...

I noticed you referenced 'boinc --daemon', where as I have been using .boinc_client'. Is there a reason to prefer one over the other?
ID: 61773 · Report as offensive
BobCat13

Send message
Joined: 6 Dec 06
Posts: 118
United States
Message 61775 - Posted: 21 Apr 2015, 14:37:20 UTC - in response to Message 61773.  

I really don't know as I have never used the .boinc_client option. I have a separate partition just for boinc and have two directories, one for the executables and another for the data. I just copy boinc, boincmgr, and boinccmd into the executables directory so that is why I use .boinc as I don't have boinc_client in that directory.
ID: 61775 · Report as offensive
BobCat13

Send message
Joined: 6 Dec 06
Posts: 118
United States
Message 61776 - Posted: 21 Apr 2015, 14:48:22 UTC - in response to Message 61774.  

so now get fancy and start a 3 copy of boinc using another port, same format as the 2nd

Hmmm, if each client sees 128 processors and tries to run 50 tasks, then you could have 150 tasks running on 128 processors. To limit it to exactly 128 tasks with 3 client instances, you could create preferences for home with 34% of total CPUs and school with 33% of total CPUs. Assign two of the instances to the home profile and one to the school profile.

That would create 2 instances with 43 threads and 1 with 42 threads. This would allow each client to have a few tasks in ready to start state as a buffer to cut down on any idle time while requesting more tasks from the MW server.
ID: 61776 · Report as offensive
usao
Avatar

Send message
Joined: 15 Apr 15
Posts: 50
United States
Message 61778 - Posted: 21 Apr 2015, 15:01:39 UTC - in response to Message 61777.  

so now get fancy and start a 3 copy of boinc using another port, same format as the 2nd

Hmmm, if each client sees 128 processors and tries to run 50 tasks, then you could have 150 tasks running on 128 processors. To limit it to exactly 128 tasks with 3 client instances, you could create preferences for home with 34% of total CPUs and school with 33% of total CPUs. Assign two of the instances to the home profile and one to the school profile.

That would create 2 instances with 43 threads and 1 with 42 threads. This would allow each client to have a few tasks in ready to start state as a buffer to cut down on any idle time while requesting more tasks from the MW server.


yep, that or could do a cc_config.xml for each data directory and use ncpu to split up the threads(cpus)

must be some strangeness with this computer, os, and mw@h that has hit a 50 thread limit. new one on me, never seen this before....



Ill let it run with the 98 threads for a while and make sure the "credits" are working as expected. (that's the only unit-of-measure that I know of currently).

I may try 4 instances of 32 each, to get to 128. I see the option in the XML files to set the number of cpu's, so that could work.
ID: 61778 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15482
Netherlands
Message 61781 - Posted: 21 Apr 2015, 16:47:20 UTC - in response to Message 61778.  

I see the option in the XML files to set the number of cpu's, so that could work.

If by that you mean the <ncpus> option, that's for debugging only. With this option you can tell BOINC to use more CPU cores than you actually have. So you can tell it to run 14 instances of work on a quad core.

'Normal' CPU preferences are set through the "Use at most N% of the processors", or in Milkyway's preferences name, "On multiprocessors, use at most N% of the processors".
ID: 61781 · Report as offensive
Coleslaw
Avatar

Send message
Joined: 23 Feb 12
Posts: 198
United States
Message 61795 - Posted: 21 Apr 2015, 18:50:54 UTC - in response to Message 61778.  

so now get fancy and start a 3 copy of boinc using another port, same format as the 2nd

Hmmm, if each client sees 128 processors and tries to run 50 tasks, then you could have 150 tasks running on 128 processors. To limit it to exactly 128 tasks with 3 client instances, you could create preferences for home with 34% of total CPUs and school with 33% of total CPUs. Assign two of the instances to the home profile and one to the school profile.

That would create 2 instances with 43 threads and 1 with 42 threads. This would allow each client to have a few tasks in ready to start state as a buffer to cut down on any idle time while requesting more tasks from the MW server.


yep, that or could do a cc_config.xml for each data directory and use ncpu to split up the threads(cpus)

must be some strangeness with this computer, os, and mw@h that has hit a 50 thread limit. new one on me, never seen this before....



Ill let it run with the 98 threads for a while and make sure the "credits" are working as expected. (that's the only unit-of-measure that I know of currently).

I may try 4 instances of 32 each, to get to 128. I see the option in the XML files to set the number of cpu's, so that could work.


usao, are you only running the one project? Some projects do have a hard limit on total number of tasks they will allow a box to run. CAS for example seems to only allow up to 40 at a time (last my teammates checked). So, it is feasable as mentioned above that it is the cause of this problem. So, you are certainly doing what is necessary to get around it without using VM's. However, if you run other projects, that might allow you to use all of the cores/threads without managing multiple clients.

Please confirm if you get the itch.
ID: 61795 · Report as offensive
usao
Avatar

Send message
Joined: 15 Apr 15
Posts: 50
United States
Message 61797 - Posted: 21 Apr 2015, 19:14:21 UTC - in response to Message 61795.  

I haven't really looked at other projects yet. I only looked at SETI and MilkyWay so-far. Ill have to read up on the others to see if they are interesting.

Im always worried that there could be some bogus project run by the CIA to hack the world, and disguised as a real project...
ID: 61797 · Report as offensive

Message boards : Questions and problems : Cant figure out how to get more than 50 threads running

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.