PCI express risers to use multiple GPUs on one motherboard - not detecting card?

Message boards : GPUs : PCI express risers to use multiple GPUs on one motherboard - not detecting card?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 20 · Next

AuthorMessage
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96168 - Posted: 28 Feb 2020, 13:52:13 UTC - in response to Message 96155.  

i'm running mine on Linux as well.

2080ti default power limit is usually around 250W. what model do you have that is 300W?

additionally, you can regain some lost performance from power limiting by just adding an overclock. I power limit + overclock all of my cards for a boost in efficiency and minimal performance hit.

RTX 2080s, limited from 215W -> 200W, +100 core +400 memory
RTX 2070s, limited from 175W -> 165W, +100 core +400 memory
ID: 96168 · Report as offensive
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96169 - Posted: 28 Feb 2020, 14:09:15 UTC - in response to Message 96155.  
Last modified: 28 Feb 2020, 14:09:44 UTC

looking at your run times, looks like you are getting a pretty serious performance hit. you can tell which ones are on the PCIe riser, and which have more bandwidth. just look at the ones with mismatched CPU vs run time, and earning less credit per time spent crunching.

Yours: http://www.gpugrid.net/results.php?hostid=516995
Miklos' 2080ti: http://www.gpugrid.net/results.php?hostid=522246

My 2080s at 200W even perform better:
https://www.gpugrid.net/results.php?hostid=524248
ID: 96169 · Report as offensive
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96181 - Posted: 28 Feb 2020, 23:16:47 UTC - in response to Message 96178.  
Last modified: 28 Feb 2020, 23:20:30 UTC

The core overclock just brings the clocks back to where they would be without the power limit.

The memory overclock just puts them back to the P0 clocks. Nvidia artificially limits their consumer line cards memory clocks to P2 when under compute loads. But they allow full clock speed on gaming loads. This started around the Maxwell generation I think. So a stock non-overclocked 1080ti for example will run P0 in games (11,000MHz effective), but be stuck on P2 for compute (10,000MHz effective). Adding an “overclock” to this level isn’t really an overclock in my eyes, just putting back what Nvidia took away. It’s not any more stressful than playing games would be. Nvidia doesn’t impose this restriction on their Quadro line cards. You can overclock further of course but I don’t.

Some of the lower end cards like 1050 or 1050ti don’t have this restriction for some reason either.

I have my cards like this for over a year at this point. It’s very conservative and safe. Any card that fails from this kind of operation was defective and would have failed anyway.
ID: 96181 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 718
United States
Message 96193 - Posted: 29 Feb 2020, 11:03:26 UTC - in response to Message 96168.  
Last modified: 29 Feb 2020, 11:14:03 UTC

i'm running mine on Linux as well.

2080ti default power limit is usually around 250W. what model do you have that is 300W?

additionally, you can regain some lost performance from power limiting by just adding an overclock. I power limit + overclock all of my cards for a boost in efficiency and minimal performance hit.

RTX 2080s, limited from 215W -> 200W, +100 core +400 memory
RTX 2070s, limited from 175W -> 165W, +100 core +400 memory


I run Gigabyte Windforce (3 FAN) GPUs (300W max).
Over the years, I've moved everything (EVGA, ASUS, MSI, Zotac) all to Gigabyte. Never had a single one of them go bad.
Not sure if it's because they all have a back plate. But other brand GPUs would die sporadically every few months.

I usually run the GPU at 125Mhz overclock on those (which says very little, as my EVGA GPUs did +205Mhz to reach the same boost clock rates), and 1400Mhz memory; which after a lot of testing, I found to be the sweet spot for all RTX GPUs (ranging from 2060 to 2080Ti, and between all brands);
The memory modules are 16Gbits modules anyway, and +1400Mhz OC (14.900Mhz) is in the middle between stock (13.500Mhz) and max (16.000)overclock. (max overclock is guarantee failures during compute).

I run them in an open bench, at between 199W and 225W. For Folding, and some projects like Einstein, I could run them at 150W, max GPU frequency of 2100Mhz.
But for Collatz and Core 22 folding, 199W is the lowest I would go (as even with an overclock they barely can do 1750Mhz).
I don't run them beyond 225W, as for smaller projects (like Einstein) they would hit past their max rated GPU frequency (sometimes 2200-2400Mhz) and crash the task; or go into limp mode (1350Mhz) and do it very slow.

Power capping is the opposite of overclocking. you're actually slowing the GPU down.
There is zero possibility for you to blow your GPU by power capping.
Overclocking on the other hand, only when you know what you're doing.
I had FAH to help me find the optimal overclocking settings on my GPUs, and barely have a single fail.

The reason I don't run a lot of GPUGrid, is that their tasks are more aimed towards RTX 2060/2070 or below. They don't fully use the RTX 2080 GPUs or higher.
There are 2 scenarios that make a 2080Ti slower than a stock 2080:
1- The Ti stock, is only taxed 50-70% on those tasks. This is in line with a 2070/2080 in amount of active cores, but a 2080 Ti has a lower core clock most of the time under load.
2- The Ti will run into Limp mode once the load gets too low, or the overclock hits too high GPU frequency (it'll run at 1350Mhz). Both of these cause a 2080 Ti to be less than optimal.
Same on Einstein.
In Einstein, I can run 2 projects per GPU, and see a 100% load; and I would recommend it for 2080, 2080 Super, and 2080Ti to use these settings.
That way, they'll process the task slightly slower than a 2060 would, but could do 2 or 3 at a time.
ID: 96193 · Report as offensive
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96204 - Posted: 29 Feb 2020, 14:12:27 UTC - in response to Message 96193.  

I run Gigabyte Windforce (3 FAN) GPUs (300W max).
Over the years, I've moved everything (EVGA, ASUS, MSI, Zotac) all to Gigabyte. Never had a single one of them go bad.
Not sure if it's because they all have a back plate. But other brand GPUs would die sporadically every few months.

probably just a fluke. I run mostly all EVGA cards, they are my preferred brand. (currently have 17 total across 2 systems). I've only RMA'd 1 or 2 cards. keep in mind that early on there was a problem with some RTX cards from every manufacturer. I think they had some defective memory or something. there was a lot of chatter about high RMA rates.

I usually run the GPU at 125Mhz overclock on those (which says very little, as my EVGA GPUs did +205Mhz to reach the same boost clock rates), and 1400Mhz memory; which after a lot of testing, I found to be the sweet spot for all RTX GPUs (ranging from 2060 to 2080Ti, and between all brands);
The memory modules are 16Gbits modules anyway, and +1400Mhz OC (14.900Mhz) is in the middle between stock (13.500Mhz) and max (16.000)overclock. (max overclock is guarantee failures during compute).

the default (P0) memory speed for almost all RTX cards including your 2080ti is 14,000MHz, with the only exception being the 2080 Super which has a default speed of 15,500MHz. stock compute loads (P2) gets a 400MHz penalty, 13,600MHz. so you overclocking to 14,000 is exactly what I do, just bring it back to P0 clocks.

I run them in an open bench, at between 199W and 225W. For Folding, and some projects like Einstein, I could run them at 150W, max GPU frequency of 2100Mhz.
I don't run them beyond 225W, as for smaller projects (like Einstein) they would hit past their max rated GPU frequency (sometimes 2200-2400Mhz) and crash the task; or go into limp mode (1350Mhz) and do it very slow.

i've never had this happen with Einstein. I've found that Einstein Gamma Ray tasks are quite easy on the card in general, and I do not have to change the power limit since it doesnt hit the power cap anyway, so lowering it in this scenario has no effect. it only uses about 125-130W or so on my 2070's when running gamma ray tasks. core clocks stay about 2000MHz, which is normal for these low end 2070 cards (2070 Black, cheapest EVGA model).

another thing that i do for stability is to run a small script in the background which places a tiny load on each GPU. this holds the GPU in the P2 state. when you overclock, it gets applied to both the P0 and P2 states. when tasks start and stop you can see momentary states toggle from P2 to P0. that momentary switch can causing crashing from the too high overclock. it was more important when I was running 1080tis with a +1000 memory overclock, since the momentary switch to P0 would induce a 12,000MHz clock speed. maybe it's not as necessary anymore with the smaller penalty for RTX cards, but I stil run it anyway for piece of mind.

Power capping is the opposite of overclocking. you're actually slowing the GPU down.
There is zero possibility for you to blow your GPU by power capping.
Overclocking on the other hand, only when you know what you're doing.


power capping does not slow you down. it just lowers one of the many performance cap variables (others include voltage and temperature). if you hit the cap, you can be prevented from going further. you can "go faster" even with a power cap by just applying an overclock on top of it. take a card with no overclock, then power cap it. then add an overclock, boom you just sped up despite the cap. the card will try to accommodate to give you more speed within the power limits, by adjusting voltage down.

the only time the card will actually slow you down is if it can't control the temperatures.

its almost impossible to hurt a modern nvidia GPU from overclocking. there are so many limits and safety features in place to protect the card. take any RTX GPU, and crank all the settings to the max. it might crash, it might be unstable, but you wont hurt the card from it. you really need to go to extremes like custom modifying the BIOS with higher power/voltage limits, or physically modifying the card to actually cause damage from an overclock.

The reason I don't run a lot of GPUGrid, is that their tasks are more aimed towards RTX 2060/2070 or below. They don't fully use the RTX 2080 GPUs or higher.
There are 2 scenarios that make a 2080Ti slower than a stock 2080:
1- The Ti stock, is only taxed 50-70% on those tasks. This is in line with a 2070/2080 in amount of active cores, but a 2080 Ti has a lower core clock most of the time under load.
2- The Ti will run into Limp mode once the load gets too low, or the overclock hits too high GPU frequency (it'll run at 1350Mhz). Both of these cause a 2080 Ti to be less than optimal.
Same on Einstein.
In Einstein, I can run 2 projects per GPU, and see a 100% load; and I would recommend it for 2080, 2080 Super, and 2080Ti to use these settings.
That way, they'll process the task slightly slower than a 2060 would, but could do 2 or 3 at a time.


most of the fastest hosts on GPUGrid are using RTX 2080tis. just look at the link to Miklos' system. i've not seen any people talk about low load on high power cards in Linux (only Windows). my 7x2080 system runs GPUGrid fine right at ~97% on all cards. And just to check, I booted my gaming machine (9700k/2080ti) into linux, and started up a GPUGrid job and it also runs just fine at 97%, tried both at stock 250W power limit as well as 200W. there's something wrong with your setup if it's only running 50-60%. probably PCIe bandwidth if on a USB riser or not enough CPU.

Proof: https://imgur.com/a/NTirSri
ID: 96204 · Report as offensive
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96215 - Posted: 29 Feb 2020, 19:31:21 UTC - in response to Message 96213.  

the default (P0) memory speed for almost all RTX cards including your 2080ti is 14,000MHz, with the only exception being the 2080 Super which has a default speed of 15,500MHz. stock compute loads (P2) gets a 400MHz penalty, 13,600MHz. so you overclocking to 14,000 is exactly what I do, just bring it back to P0 clocks.


A 2080Ti has 1750MHz memory. 14000MHz is just an "effective speed" (as in sales jargon). No memory chip can reach 14GHz.


yes, if you look back at my previous replies you can see i used "effective" when referencing mem speeds. it's just pedantic to keep having to add that when everyone understands what is being discussed. 14,000MHz effective, or if you prefer, 14Gbps
ID: 96215 · Report as offensive
Nick Name

Send message
Joined: 14 Aug 19
Posts: 55
United States
Message 96227 - Posted: 1 Mar 2020, 4:54:31 UTC - in response to Message 96193.  


I don't run them beyond 225W, as for smaller projects (like Einstein) they would hit past their max rated GPU frequency (sometimes 2200-2400Mhz) and crash the task; or go into limp mode (1350Mhz) and do it very slow.

Why not just buy a lower power card if you're going to limit the power so severely?
Team USA forum
Follow us on Twitter
Help us #crunchforcures!
ID: 96227 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 718
United States
Message 96229 - Posted: 1 Mar 2020, 5:40:00 UTC - in response to Message 96227.  
Last modified: 1 Mar 2020, 5:47:49 UTC

Just for reference, the memory modules on all RTX GPUs are rated at ~14Gbps, while the Supers are rated at ~15Gbps. Though they all are the same modules. The Samsung and hynix modules are 15Gbps modules that can reach a theoretical maximum (with overclock) of 16Gbps, though anywhere above 15-15.5Gbps there's no substantial performance improvement, as well as an increase in glitches.

The super GPUs are just rated higher, as the standard RTX GPU memory speed was underrated.



I don't run them beyond 225W, as for smaller projects (like Einstein) they would hit past their max rated GPU frequency (sometimes 2200-2400Mhz) and crash the task; or go into limp mode (1350Mhz) and do it very slow.

Why not just buy a lower power card if you're going to limit the power so severely?


One, because the higher end GPUs don't need to run at their rated power to do the tasks.
Two, because higher end GPUs have more cores.
Three, because their performance is nearly identical (with overclock) at 20-60% power cap, vs stock.
Running at stock wattage, the GPU frequency is limited. By cutting power and overclocking, you can get close to this performance, at a fraction of the power. If you run 3 GPUs, capped at 35-40% power cap, you can add a fourth one and still use the same amount of power at the wall, but with 25% extra performance.
And four, doubling up on Einstein jobs on a high end GPU, is an easy way to fully utilize the GPU.

Using a smaller GPU will perform a lot slower. For instance, the 2080Ti is about twice as fast as a 2060 on low intensity tasks. The 2060 uses 125W out of 170W stock, the 2080 Ti does it at 150W. Multiply the 125W by 2, to get similar performance as the 2080 Ti, and it's easy to see how the 2080Ti is much more efficient (at similar performance levels).

For high intensity tasks, or two tasks per GPU, a 2060 will run at 140-150W, while a 2080 Ti does it at 190-220W.
Usually high intensity tasks also have qrb (quick return bonus), which seem to favor higher end GPUs over multiple lower speed ones.
ID: 96229 · Report as offensive
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96236 - Posted: 1 Mar 2020, 14:34:14 UTC - in response to Message 96229.  

The super GPUs are just rated higher, as the standard RTX GPU memory speed was underrated.


No. ONLY the 2080 Super is rated to 15.5Gbps. The 2070 super and 2060 super are still rated to 14Gbps.

even they are the "same" chips from the same factories, there is such a thing as binning and not all chips will be as stable at higher speeds.
ID: 96236 · Report as offensive
Profile Keith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 863
United States
Message 96243 - Posted: 1 Mar 2020, 19:49:16 UTC - in response to Message 96181.  

Some of the lower end cards like 1050 or 1050ti don’t have this restriction for some reason either.

Because Nvidia does not see those cards as a threat to their bottom line for replacement of their compute class Quadro and Tesla product lines.

They do see the top end consumer cards as stealing from the bottom line of the Quadros and Teslas and so artificially limit the memory clocks on consumer cards whenever a compute load is detected by the drivers.

No other reason than economics and greed.
ID: 96243 · Report as offensive
Nick Name

Send message
Joined: 14 Aug 19
Posts: 55
United States
Message 96245 - Posted: 1 Mar 2020, 20:18:41 UTC - in response to Message 96229.  

One, because the higher end GPUs don't need to run at their rated power to do the tasks.
Two, because higher end GPUs have more cores.
Three, because their performance is nearly identical (with overclock) at 20-60% power cap, vs stock.
Running at stock wattage, the GPU frequency is limited. By cutting power and overclocking, you can get close to this performance, at a fraction of the power. If you run 3 GPUs, capped at 35-40% power cap, you can add a fourth one and still use the same amount of power at the wall, but with 25% extra performance.
And four, doubling up on Einstein jobs on a high end GPU, is an easy way to fully utilize the GPU.

Using a smaller GPU will perform a lot slower. For instance, the 2080Ti is about twice as fast as a 2060 on low intensity tasks. The 2060 uses 125W out of 170W stock, the 2080 Ti does it at 150W. Multiply the 125W by 2, to get similar performance as the 2080 Ti, and it's easy to see how the 2080Ti is much more efficient (at similar performance levels).

For high intensity tasks, or two tasks per GPU, a 2060 will run at 140-150W, while a 2080 Ti does it at 190-220W.
Usually high intensity tasks also have qrb (quick return bonus), which seem to favor higher end GPUs over multiple lower speed ones.

I'll have to revisit this as the last time I spent any significant time testing this was on Pascal 1080 and this kind of performance wasn't possible. Performance started to degrade more than I was willing to tolerate around an 80% power limit and I was not able to recover any by overclocking. Even a 10 mhz overclock at that limit sent invalids skyrocketing. I have limited a Radeon VII to this extent but there I have control over voltage which helped stabilize things.
Team USA forum
Follow us on Twitter
Help us #crunchforcures!
ID: 96245 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 718
United States
Message 96257 - Posted: 2 Mar 2020, 3:51:48 UTC - in response to Message 96236.  

The super GPUs are just rated higher, as the standard RTX GPU memory speed was underrated.


No. ONLY the 2080 Super is rated to 15.5Gbps. The 2070 super and 2060 super are still rated to 14Gbps.

even they are the "same" chips from the same factories, there is such a thing as binning and not all chips will be as stable at higher speeds.


All (note *ALL*) RTX GPUs run stable at 15GBPS.
ID: 96257 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 718
United States
Message 96258 - Posted: 2 Mar 2020, 3:57:25 UTC - in response to Message 96247.  
Last modified: 2 Mar 2020, 4:06:21 UTC

One, because the higher end GPUs don't need to run at their rated power to do the tasks.
Two, because higher end GPUs have more cores.
Three, because their performance is nearly identical (with overclock) at 20-60% power cap, vs stock.
Running at stock wattage, the GPU frequency is limited. By cutting power and overclocking, you can get close to this performance, at a fraction of the power. If you run 3 GPUs, capped at 35-40% power cap, you can add a fourth one and still use the same amount of power at the wall, but with 25% extra performance.
And four, doubling up on Einstein jobs on a high end GPU, is an easy way to fully utilize the GPU.


I simply can't believe this, it's rather like the perpetual motion device. You can't use less power and get more calculations done. It's against common sense.


Yet it's how I've been folding and crunching for over 1.5 years now (1 year on RTX).
A lot of the GPU is passive during crunching (eg: vram, Boinc projects usually use between 100 to 700MB max). Powercapping will disable power to unused memory modules.
It also takes a significantly more amount of power, to run the GPU from 99 to 100%, than from 90-95%. By altering the fan curve (to 100%), the GPU runs cooler and more stable at lower voltages, resulting in more than the 2 Watts invested in higher fan rotation.

So running GTX and RTX GPUs at 90% of the GPU frequency, often saves nearly 40% of the power, and affects crunching by less than 10%.
Remember, these cards often come overclocked from the factory.
The power saved, cen be reinvested in another GPU.

The dynamics of it all are quite interesting, and definitely a lot of fun for system builders, and people generally interested in hardware.
ID: 96258 · Report as offensive
Ian&Steve C.

Send message
Joined: 24 Dec 19
Posts: 228
United States
Message 96274 - Posted: 2 Mar 2020, 19:19:03 UTC - in response to Message 96272.  
Last modified: 2 Mar 2020, 19:20:33 UTC

the GPU control systems these days are more sophisticated than you think they are. And Nvidia is a lot better at it than AMD is.

Power is only a function of Voltage and Current. Current is mostly a function of clock speed. (ignoring differences such as architecture and process node).

So if you keep clock speed the same, but reduce the voltage. the GPU will be just as fast and do the same work, but using less power. the degree to which you can do this depends on how good the individual chip is on the silicon level. AMD used to refer to this kind of thing as "Silicon Quality" and you could actually get a reading of this value from older AMD chips. if you reduce the voltage too much, it becomes unstable. but usually you should be able to reduce the voltage a little bit and remain stable. Any reduction in voltage will reduce power used.

we have to play with power limits to do this on nvidia, since they all but removed voltage control from the end user on their newer cards (unless you hard mod the GPU hardware). So if you say give me X clocks at Y power limit, the card will "try" the best it can, but at some point it stops giving you more clocks since it can't/won't reduce voltage any further.
ID: 96274 · Report as offensive
Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 20 · Next

Message boards : GPUs : PCI express risers to use multiple GPUs on one motherboard - not detecting card?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.