CPU Throttling causing cuda task to not suspend when user active

Message boards : Questions and problems : CPU Throttling causing cuda task to not suspend when user active
Message board moderation

To post messages, you must log in.

AuthorMessage
Duff_Moss

Send message
Joined: 16 Jun 11
Posts: 7
Australia
Message 38493 - Posted: 16 Jun 2011, 14:01:06 UTC

Thought I'd post a solution to a problem I was having - that may be of use to others.

I was running a client on a PC with a NVIDIA card. I was only processing GPU tasks. I configured the system to suspend if the user was active, however I found that often as not, the cuda task would not suspend. It seemed completely random as to when it would or would not work correctly.

It drove me nuts for weeks and I eventually found it was the CPU throttling. I had CPU throttling set to 50% (from a former time when I was doing non cuda crunching) and I think it was creating a race condition, so that if the instruction to throttle coincided with the instruction to suspend the WU, it would sometimes get missed.

I didn't try it, but I suspect if the throttling was say use 80% of CPU, then the frequency of the failure would be less, as the client would be issuing a throttle command less frequently.

Either way, if you're having problems with tasks not suspending, and you have throttling turned on - turn it off and see if it helps you too.

I tried various BOINC versions and had the same issue.
ID: 38493 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 38495 - Posted: 16 Jun 2011, 14:59:16 UTC - in response to Message 38493.  

It also depends on what operating system you have and whether you have rebooted in the past couple of days. BOINC its idle detection on Windows Vista and Windows 7 runs through a small program called boinctray.exe.

When you install a new version of BOINC, all present files loaded into memory will be unloaded & these files will be deleted from disk by the uninstaller. Then the installer of the new BOINC puts the files back on the disk, but due to permission problems it cannot thus far restart boinctray.exe

And as such, it may well happen that you have upgraded your BOINC, that you're expecting it to follow your preferences but that it doesn't because it misses this idle detection program. It's only started from the registry, not by BOINC itself.

This is easily fixed, if you pass by on the BOINC programs directory and start the program ((double-)click on it to start it, it won't show any window or other confirmation that it's running, other than showing up in Windows Task Manager), or if you're finicky about that, reboot your system.
ID: 38495 · Report as offensive
Duff_Moss

Send message
Joined: 16 Jun 11
Posts: 7
Australia
Message 38509 - Posted: 17 Jun 2011, 4:23:53 UTC - in response to Message 38495.  

It definitely wasn't a boinctray problem. Boinctray was running.

If I watched the boinc manager task screen, it would mark all WUs as suspended, user active (or some such thing), indicating it detected user activity. But sometimes the currently running WU would just keep crunching away even though it was reported as suspended.

If I then let the computer go idle again, boincmanager would indicate the WU was running. If I then used the desktop again, bonicmanager would advise that the user was active and suspend the WUs. Here, again, sometimes the WU would actually suspend, but also sometimes would just keep going.

I turned on some debugging (can't remember which ones currently) which would show the client issuing a suspend command and a resume command. I saw that in those times that the WU kept running, the suspend command was not issued.

I then turned on some more debugging which showed the CPU throttle on and off commands, and they were issued every half a second. I noticed that when the suspend command was missed, a throttle on or off command occurred at the same time (hard to tell really due to the frequency, but it seemed that way after many observations).

When I turned the CPU throttle off, the WUs would suspend as they were supposed to EVERY time. This was a couple of months ago now and I've not had a single missed suspend since.

I'm sure there are other reasons for most people, but I looked on these boards at the time and couldn't find a resolution to my issue. I saw others in the same boat (had tried all other suggestions) and I simply offer that what I observed may help some others here too.
ID: 38509 · Report as offensive

Message boards : Questions and problems : CPU Throttling causing cuda task to not suspend when user active

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.