How to exclude multiple GPUs for a given Project, using GPU_Exclude ?

Message boards : GPUs : How to exclude multiple GPUs for a given Project, using GPU_Exclude ?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile ncoded.com

Send message
Joined: 13 Dec 16
Posts: 52
United Kingdom
Message 76720 - Posted: 23 Mar 2017, 15:49:50 UTC
Last modified: 23 Mar 2017, 16:13:30 UTC

Hi,

I have this, to use the first GPU but exclude the 2nd for a given project, but use all GPUs for the rest of the projects.

<cc_config>
<options>
<use_all_gpus>1</use_all_gpus>
<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>1</device_num>
<type>NVIDIA</type>
</exclude_gpu>
</options>
</cc_config>

However, I am about to move all 4/5 GPUs into our GPURack and therefore I was wondering how I would exclude say 3/4 GPUs from a given Project? I am guessing it would something like:

<device_num>1,2,3,4</device_num>

or

<device_num>1</device_num>
<device_num>2</device_num>
..
..

Could someone advise; I have done a search but cannot find anything related to excluding more than one GPU within a single system.

Thanks.
ID: 76720 · Report as offensive
Profile Ageless
Volunteer moderator
Project administrator
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 12666
Netherlands
Message 76722 - Posted: 23 Mar 2017, 16:22:01 UTC - in response to Message 76720.  
Last modified: 23 Mar 2017, 16:22:22 UTC

It's the latter. You tell BOINC which device it's about, per device.
But watch out when you mix GPU brands, as both main brands have their own device numbers.

So if for instance you add two Nvidia and two AMD GPUs in one computer, it's for both device 0 and device 1.
It's just that it's Nvidia device 0, Nvidia device 1, and AMD device 0, AMD device 1.
In all other cases, same brand it's device 0, device 1, device 2, device 3, device 4.
Jord
Please do not private message me for tech support. Use the forums for that. Tech PMs will be ignored.

quote: "I'm an apostrophe, I'm just a symbol to remind you that there's more to see"
ID: 76722 · Report as offensive
Richard Haselgrove
Volunteer moderator
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 2819
United Kingdom
Message 76723 - Posted: 23 Mar 2017, 16:25:38 UTC - in response to Message 76720.  

I think the instructions in the Client Manual are pretty clear. Under <exclude_gpu>, there is:

You may include multiple <exclude_gpu> elements.
- in other words, repeat everything between (and including)

<exclude_gpu>
...
</exclude_gpu>

as many times as needed, changing an item or items within each block.
ID: 76723 · Report as offensive
Richard Haselgrove
Volunteer moderator
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 2819
United Kingdom
Message 76724 - Posted: 23 Mar 2017, 16:28:08 UTC

Um, or maybe not. Multiple blocks, or multiple lines within each block? Try Jord's answer first, he wrote the FAQs... ;-)
ID: 76724 · Report as offensive
Profile Ageless
Volunteer moderator
Project administrator
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 12666
Netherlands
Message 76727 - Posted: 23 Mar 2017, 20:10:38 UTC - in response to Message 76724.  

Try Jord's answer first, he wrote the FAQs... ;-)
Sure... blame me. :)
Jord
Please do not private message me for tech support. Use the forums for that. Tech PMs will be ignored.

quote: "I'm an apostrophe, I'm just a symbol to remind you that there's more to see"
ID: 76727 · Report as offensive
Profile ncoded.com

Send message
Joined: 13 Dec 16
Posts: 52
United Kingdom
Message 76733 - Posted: 24 Mar 2017, 1:16:19 UTC
Last modified: 24 Mar 2017, 1:18:39 UTC

Hi Jord, Richard.

Thanks for the information.

I was not sure where the manual was located so it good that I now have this.

http://boinc.berkeley.edu/wiki/Client_configuration#Options

Although I would agree that the entry it is a little unclear it respects to different GPU brands, and having to put in multiple <exclude_gpus>, one for each device when excluding multiple GPUs; Perhaps another two examples showing these variants would be useful. (as given below).

--

So just to confirm, for myself, and others..

I would use (a) to use the 1st GPU for GPUGrid, but exclude the remaining 3 GPUs, however also have all GPUs used for every other project?

And, use (b) to do the same, but in the case where the first 2 GPUs were NVidia, and the remaining 2 GPUs were ATI?

(Remembering of course that GPUs are numbered from zero, not one; and as you stated that each brand: NVidia/ATI/Intel has their own number list)

(a)

<cc_config>
<options>
<use_all_gpus>1</use_all_gpus>

<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>1</device_num>
</exclude_gpu>

<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>2</device_num>
</exclude_gpu>

<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>3</device_num>
</exclude_gpu>

</options>
</cc_config>

(b)

<cc_config>
<options>
<use_all_gpus>1</use_all_gpus>

<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>1</device_num>
<type>NVIDIA</type>
</exclude_gpu>

<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>0-</device_num>
<type>ATI</type>
</exclude_gpu>

<exclude_gpu>
<url>www.gpugrid.net</url>
<device_num>1</device_num>
<type>ATI</type>
</exclude_gpu>

</options>
</cc_config>

--

Just as a side issue, just so you know, when subscribed to a thread, such as this one, no notification of new posts are being sent.
ID: 76733 · Report as offensive
Profile ncoded.com

Send message
Joined: 13 Dec 16
Posts: 52
United Kingdom
Message 76736 - Posted: 24 Mar 2017, 4:23:19 UTC

Where is the edit button for my last post?

(Remembering of course that GPUs are numbered from zero, not one; and as you stated that each brand: NVidia/ATI/Intel has their own number list)


This is wrong..

GPUs are numbered according to:

Tools/Event Log - from the top menu within the BOINC client.
ID: 76736 · Report as offensive
Profile Ageless
Volunteer moderator
Project administrator
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 12666
Netherlands
Message 76745 - Posted: 24 Mar 2017, 8:40:10 UTC - in response to Message 76733.  

a) and b) are correctly set up. Although in the case of a) it won't hurt to specify the GPU brand, makes it more readable for the future as well.
Yes, GPU numbers start from 0. That's a hardware number, not BOINC specific. Usually the GPU in the slot closest to the output connectors on the back, or using the top PCIe slot in a tower, is device 0.
Intel GPUs are always device 0, there's no further numbering because Intel GPUs are built into the CPU and till thus far there's only one GPU built in.

Editing posts is only possible for the duration of one hour after you posted something, or until the sun dies when you're a moderator. :)
Jord
Please do not private message me for tech support. Use the forums for that. Tech PMs will be ignored.

quote: "I'm an apostrophe, I'm just a symbol to remind you that there's more to see"
ID: 76745 · Report as offensive
Profile ncoded.com

Send message
Joined: 13 Dec 16
Posts: 52
United Kingdom
Message 76749 - Posted: 24 Mar 2017, 12:34:15 UTC
Last modified: 24 Mar 2017, 12:35:34 UTC

We only use NVidia GPUs, and always turn off Intel IGPUs across all projects, so its less of an issue for us.
it won't hurt to specify the GPU brand


What a I would say on the GPU numbeing. is on some mobos/gpus it is different; which is why I guess you said *usually.
In our case, the current setup is:

    ASUS X99-Deluxe Motherboard (LGA 2011-V3)
    GPUs are a MSI 1080 and a Asus 970.



The 970 is in slot 1, and the 1080 is in slot 2. However the event log shows that the 1080 is GPU 0, and the 970 is GPU 1.
That is why I noted that it is probably best to check the Event Log to see exactly which number each GPU has been allocated.

Anyway, thanks for all the help and information, its much appreciated.

ID: 76749 · Report as offensive
Profile Ageless
Volunteer moderator
Project administrator
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 12666
Netherlands
Message 76750 - Posted: 24 Mar 2017, 13:29:39 UTC - in response to Message 76749.  
Last modified: 24 Mar 2017, 13:33:38 UTC

What a I would say on the GPU numbering. is on some mobos/gpus it is different; which is why I guess you said *usually.
That's in case the motherboard or CPU comes with a built-in GPU, which both Intel and AMD now do. But where the Intel GPU is then just device 0 of that brand, in the case of AMD APUs, I would expect the built-in Radeon to be device 0 and any next GPU of that brand to be device 1, 2 etc.

Of course, an APU won't have a massive motherboard that allows for plugging in of multiple cards. Whereas motherboards with a built-in GPU are becoming scarce because of the built-in ones in CPUs.

Also when you plug in an Nvidia in this case, whether or not the built-in AMD/other brand GPU is device 0 is moot, because whichever Nvidia is first is that brand's device 0. :)

And motherboards with a version of an Nvidia GPU built-in, normally don't allow for plugging of anything else in there. E.g. ION based motherboards.
Jord
Please do not private message me for tech support. Use the forums for that. Tech PMs will be ignored.

quote: "I'm an apostrophe, I'm just a symbol to remind you that there's more to see"
ID: 76750 · Report as offensive
Profile ncoded.com

Send message
Joined: 13 Dec 16
Posts: 52
United Kingdom
Message 76752 - Posted: 24 Mar 2017, 17:34:05 UTC
Last modified: 24 Mar 2017, 17:47:49 UTC

Sorry you lost me a little bit there :?

When we move in/out all the GPUs I shall just take a look at the Event log to find out which GPU is numbered which.

In terms of the IGPU, because we are using a Xeon CPU it means there is no on-board IGPU.

To be honest we stopped using the IGPUs on our i7s as it seemed to overheat the CPUs when running at 100% load for weeks whilst using using stock Intel coolers.

At the moment we are just testing out the new GTX 1080 on this motherboard hence why we only have 2 NVIdia GPUs in place; in the next day or so though I am going to move this motherboard into the GPURack and then hook up probably 4 GPUs.

After months of research we went with a X99 LGA-2011 V3 motherboard due to as explained below..

--

In terms of the GPU Rack, just to explain, and in-case anyone is interesting in how we set it up and what we used:

We went with ASUS X99 Deluxe motherboard + Xeon E5-2683 V3 CPU (28 cores) as it is perfect for a GPU Rack.

The X99 Deluxe can:

    Take V3 Xeon CPUs, which have 40 PCIE lanes
    Has 4 PCIE slots (x16 size) at Gen3 with speeds (x16,x8,x8,x8) or 5 PCIE slots (x16 size) at Gen3 with speeds (x8.x8.x8.x8,x8)
    Can "overclock" (full Turbo) the CPU just by clicking a button in the BIOS (OCGenie); make sure you pick the correct RAM speed for the Xeon.
    Has an ATX Form Factor size which fits the GPU Rack
    Takes DDR4 RAM which is obviously much better than DDR3



To make sure we keep the full x16/x8 speed we are using x16 to x16 (molex powered) PCIE risers so that we get the full x16 data bus from motherboard to GPU.

** Note - The PCIE slots above, at those speeds, are only available for Xeons, due to the 40 PCIE lanes, not i7's etc which only have 16 PCIE lanes. **

** Note - The X99 Deluxe actually has 6 GPU slots, however if you use all them then you would not get the PCIE speeds as given above, which may or many not have relevance dependent on which project you crunch as well as what GPU(s) you use.

--

If I get a chance I will take a photo of the GPU Rack once is built in a day or so, and post it as it looks quite good :)

ID: 76752 · Report as offensive

Message boards : GPUs : How to exclude multiple GPUs for a given Project, using GPU_Exclude ?

Copyright © 2018 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.