Is there a "device_nums" for app_config?

Message boards : Questions and problems : Is there a "device_nums" for app_config?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Joseph Stateson
Volunteer tester
Avatar

Send message
Joined: 27 Jun 08
Posts: 641
United States
Message 84434 - Posted: 19 Jan 2018, 6:40:48 UTC
Last modified: 19 Jan 2018, 6:41:13 UTC

Looking through the wiki here, I do not see anything like <device_number> or <device_nums> for app_config.xml

I have two similar nvidia boards but one had 1/2 the memory the other one has. I can run 4 concurrent tasks from a certain project on the larger board but only 2 concurrent on the smaller board from that same project (they would all error out if 4 tried to run).

I do not see how to do that. The best I can think of is to exclude the project from the smaller board and use another project instead

for example: I want 4 Einstein on the 4gb board but only 2 on the 2gb board.

what I managed to do was 4 Einstein on the larger and 2 Milkyway on the smaller. I had to exclude each project from the corresponding "other" board.
ID: 84434 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 84437 - Posted: 19 Jan 2018, 18:10:45 UTC - in response to Message 84434.  

The best I can think of is to exclude the project from the smaller board and use another project instead


I believe that is the best that can be done with the available options.
ID: 84437 · Report as offensive
floyd
Help desk expert

Send message
Joined: 23 Apr 12
Posts: 77
Message 84439 - Posted: 19 Jan 2018, 20:33:41 UTC - in response to Message 84434.  

I want 4 Einstein on the 4gb board but only 2 on the 2gb board.
Make sure the 4GB board is #0, then use app_config to set a maximum of 6 concurrent tasks. If you're lucky #0 is filled first, resulting in a 4+2 distribution.
ID: 84439 · Report as offensive
Profile Joseph Stateson
Volunteer tester
Avatar

Send message
Joined: 27 Jun 08
Posts: 641
United States
Message 84443 - Posted: 20 Jan 2018, 11:13:23 UTC - in response to Message 84439.  
Last modified: 20 Jan 2018, 11:14:39 UTC

I want 4 Einstein on the 4gb board but only 2 on the 2gb board.
Make sure the 4GB board is #0, then use app_config to set a maximum of 6 concurrent tasks. If you're lucky #0 is filled first, resulting in a 4+2 distribution.



This worked but I could not use it as I only had 4 cores and 6 einstein was too many for this system.

So I set it to max of 4 concurrent tasks and ran into a scheduler problem with milkyway
When the last milkyway job completes it reports nvidia job que is full (or some such wording) and no more milkyway jobs are downloaded. If I temporarily suspend einstein then milkyway downloads a boatload of tasks but eventually they are all processed and the second video board is not being used.

I think this is a bug in the scheduler as the other project is not aware the first one is limited to only 4 tasks and thinks the gpu queue is full

I ended up excluding einstein from the smaller board. The 2 additional milkyway tasks consumes very little cpu so this system works ok with 4+2
ID: 84443 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 84444 - Posted: 20 Jan 2018, 11:23:43 UTC - in response to Message 84443.  
Last modified: 20 Jan 2018, 11:23:55 UTC

I ended up excluding einstein from the smaller board.
You could also try to exclude the board for a specific application.

<exclude_gpu>
   <url>project_URL</url>
   [<device_num>N</device_num>]
   [<type>NVIDIA|ATI|intel_gpu</type>]
   [<app>appname</app>]
</exclude_gpu>

<code>app</code> specifies the short name of an application (i.e. the <code>name</code> element within the <code>app</code> element in client_state.xml). If specified, only tasks for that app are excluded.
ID: 84444 · Report as offensive
Coleslaw
Avatar

Send message
Joined: 23 Feb 12
Posts: 198
United States
Message 84477 - Posted: 22 Jan 2018, 15:30:07 UTC

You could also try using multiple clients. However, someone else would have to chime in on the best way to configure the GPU's for each client as I've not done that one yet.
ID: 84477 · Report as offensive

Message boards : Questions and problems : Is there a "device_nums" for app_config?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.