Too many WUs were downloaded, all settings are ignored

Message boards : Questions and problems : Too many WUs were downloaded, all settings are ignored
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111333 - Posted: 18 Mar 2023, 17:52:28 UTC
Last modified: 18 Mar 2023, 17:57:44 UTC

Hello,
I get on 2 of my PCs for the 2 projects i actually run too many WUs.
There are 32 CPU threads and I have setup app_config.xml to limit 9x Asteroids and 20x TNgrid running at the same time.
The Asteroids WUs are dropping down the CPU clock when I allow more then 12.
The Boinc Manager is set to 0.01 days work. And even both projects are set to Prio 0.
Both projects request work again and again until the limit of the projects is reached.
I run Ubuntu 20.04LTS with BM 7.16.6 which is the latest one for Ubuntu20.
On the GPU I run Einstein and here everything works as expected, I also have a app_config.xml
I also tried to limit the work buffer on the project homepages, no effect.
Any ideas what is wrong?

Asteroids:
<app_config>
  <app>
   <name>period_search</name>
   <max_concurrent>9</max_concurrent>
  </app>
</app_config>

TNgrid:
<app_config>
  <app>
   <name>gene_pcim</name>
   <max_concurrent>20</max_concurrent>
  </app>
</app_config>

Einstein:
<app_config>
   <app>
   <name>hsgamma_FGRPB1G</name>
   <max_concurrent>2</max_concurrent>
     <gpu_versions>
         <gpu_usage>0.5</gpu_usage>
         <cpu_usage>1</cpu_usage>
     </gpu_versions>
   </app>
</app_config>
[/code]
ID: 111333 · Report as offensive
robsmith
Volunteer tester
Help desk expert

Send message
Joined: 25 May 09
Posts: 1286
United Kingdom
Message 111334 - Posted: 18 Mar 2023, 19:50:24 UTC - in response to Message 111333.  
Last modified: 18 Mar 2023, 19:50:55 UTC

How much memory on the affected computer(s)?
Which version of BOINC?

It is not uncommon when running on a computer with too little memory to have quite significant reductions in overall performance of the computer as the operating system struggles to keep all the tasks running by swapping tasks in and out of memory.
ID: 111334 · Report as offensive
robsmith
Volunteer tester
Help desk expert

Send message
Joined: 25 May 09
Posts: 1286
United Kingdom
Message 111335 - Posted: 18 Mar 2023, 19:58:09 UTC - in response to Message 111334.  

Add to above:
While setting the cache to al low value may sound to be a good way to reduce the load on your computer it doesn't work that way. The cache is calculated from the average task duration and the number of CPU threads being used. So, if you have a 32 thread CPU and have your cache set to 1 day's worth of work you will get 32 CPU-days worth of work, and BOINC will try to run 32 tasks at a time. What you may find works is to limit the number of CPU threads BOINC can use, and not being a regular Linux user I won't suggest where to look for the appropriate file to edit to control this. However if you are using the GUI you can look under "options"/"computing preferences" and reduce the value of "use at most xx% of CPUs" to say 50%.
ID: 111335 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5082
United Kingdom
Message 111336 - Posted: 18 Mar 2023, 20:44:21 UTC

He says he's running "I run Ubuntu 20.04LTS with BM 7.16.6 which is the latest one for Ubuntu20."

By BM, I take it to mean mean the v7.16.6 package of BOINC Manager+client. He's also running app_config.xml files with <max_concurrent> entries: that combination is known to be problematic.

Short of running experimental versions, the answer is probably to watch out for v7.20.5 or v7.22.x being added to the Ubuntu repositories, and upgrade as soon as possible.
ID: 111336 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111337 - Posted: 18 Mar 2023, 21:29:41 UTC - in response to Message 111334.  

How much memory on the affected computer(s)?

32GB and most of the time only 4-5GB are used.
ID: 111337 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111338 - Posted: 18 Mar 2023, 21:38:48 UTC - in response to Message 111335.  

Add to above:
While setting the cache to al low value may sound to be a good way to reduce the load on your computer it doesn't work that way. The cache is calculated from the average task duration and the number of CPU threads being used. So, if you have a 32 thread CPU and have your cache set to 1 day's worth of work you will get 32 CPU-days worth of work, and BOINC will try to run 32 tasks at a time. What you may find works is to limit the number of CPU threads BOINC can use, and not being a regular Linux user I won't suggest where to look for the appropriate file to edit to control this. However if you are using the GUI you can look under "options"/"computing preferences" and reduce the value of "use at most xx% of CPUs" to say 50%.

If this would be the case, then I would only expect 2-3x the amount of WU to be send, not infinite.
On the other hand the value <on_frac> is around 0.5 and would compensate this.
I don't want to limit the CPU-threads to be used at all - I have a fix setting of 9 threads for Asteroids, 20 Threads for TNgrid and the rest for Einstein/Milkyway on the GPU. In summary 30-32 threads are used and shall be used.
Of course a workaround could be to start 3 Boinc instances at the same time, each for one project and there I could limit the threads and remove the app_config.xml. But I was hoping for an easier way.
ID: 111338 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111339 - Posted: 18 Mar 2023, 21:42:34 UTC - in response to Message 111336.  

He says he's running "I run Ubuntu 20.04LTS with BM 7.16.6 which is the latest one for Ubuntu20."

By BM, I take it to mean mean the v7.16.6 package of BOINC Manager+client. He's also running app_config.xml files with <max_concurrent> entries: that combination is known to be problematic.

Short of running experimental versions, the answer is probably to watch out for v7.20.5 or v7.22.x being added to the Ubuntu repositories, and upgrade as soon as possible.

I have heard that it is a bit tricky to persuade Ubuntu to work together with a newer version. But if it is needed, I would try.
Or is there an alternative for the <max_concurrent> to be used in the old BM ?
ID: 111339 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111340 - Posted: 18 Mar 2023, 21:47:06 UTC

It seems that any settings in the boinc manager about the buffer of work is overruling the settings of the project.
Is there a way that the project settings are used?
On the asteroids home page I could set the CPU threads to 33% and everything would be fine.
ID: 111340 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5082
United Kingdom
Message 111341 - Posted: 18 Mar 2023, 21:49:48 UTC - in response to Message 111339.  

I have heard that it is a bit tricky to persuade Ubuntu to work together with a newer version. But if it is needed, I would try.
Or is there an alternative for the <max_concurrent> to be used in the old BM ?
You could try

https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/boinc

I use that on Linux Mint 20.3. It doesn't install on Mint 21.1, which I think is based on Ubuntu 20.4, but that may have different security policies. If you can load it, it's been reliable for me.
ID: 111341 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2564
United Kingdom
Message 111342 - Posted: 19 Mar 2023, 8:31:03 UTC - in response to Message 111341.  

I use that on Linux Mint 20.3. It doesn't install on Mint 21.1, which I think is based on Ubuntu 20.4, but that may have different security policies. If you can load it, it's been reliable for me.
I have used Gianfranco's repository before without problems on Ubuntu but for the past two and a half years (estimate) I have been rolling my own from the master on git-hub. If you are interested in an adventure you could try compiling the release version from there yourself.
ID: 111342 · Report as offensive
robsmith
Volunteer tester
Help desk expert

Send message
Joined: 25 May 09
Posts: 1286
United Kingdom
Message 111343 - Posted: 19 Mar 2023, 10:00:11 UTC - in response to Message 111336.  

Thanks for correcting my mistakes Richard - must take more time to read and think in the future.
ID: 111343 · Report as offensive
Bryn Mawr
Help desk expert

Send message
Joined: 31 Dec 18
Posts: 287
United Kingdom
Message 111344 - Posted: 19 Mar 2023, 10:45:20 UTC - in response to Message 111339.  


I have heard that it is a bit tricky to persuade Ubuntu to work together with a newer version. But if it is needed, I would try.
Or is there an alternative for the <max_concurrent> to be used in the old BM ?


Instead of <max_concurrent> and specifying the app name you could try <project_max_concurrent> which appears to be more reliable.
ID: 111344 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111504 - Posted: 5 Apr 2023, 22:15:22 UTC - in response to Message 111344.  
Last modified: 5 Apr 2023, 22:16:03 UTC


I have heard that it is a bit tricky to persuade Ubuntu to work together with a newer version. But if it is needed, I would try.
Or is there an alternative for the <max_concurrent> to be used in the old BM ?


Instead of <max_concurrent> and specifying the app name you could try <project_max_concurrent> which appears to be more reliable.

This parameter was completely ignored.

There is some progress - Asteroids is now the only project, requesting too many WUs, TNgrid takes now care of the settings in the BM. But I have no idea why, maybe the settings on the website have higher prio.
ID: 111504 · Report as offensive
Profile Keith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 874
United States
Message 111505 - Posted: 5 Apr 2023, 23:34:24 UTC - in response to Message 111340.  

It seems that any settings in the boinc manager about the buffer of work is overruling the settings of the project.
Is there a way that the project settings are used?
On the asteroids home page I could set the CPU threads to 33% and everything would be fine.

ANY local Manager preferences ALWAYS overrides project preferences through the global_prefs_override.xml file.
If you want project preferences, toggle the "Use web preferences" button in the Manager >> Options >> Computing Preferences.
ID: 111505 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111507 - Posted: 6 Apr 2023, 7:06:45 UTC

No, I don't want that any project will override parameters.
ID: 111507 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2564
United Kingdom
Message 111509 - Posted: 6 Apr 2023, 9:41:29 UTC - in response to Message 111507.  

No, I don't want that any project will override parameters.
I Am afraid it is all or nothing. If you want only one or only some projects to override locally set parameters, you will need to go over to git-hub and put in a feature request and hope someone picks it up to work on it.
ID: 111509 · Report as offensive
Profile Keith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 874
United States
Message 111513 - Posted: 7 Apr 2023, 0:04:15 UTC - in response to Message 111507.  

No, I don't want that any project will override parameters.

Then use local preferences set in each host's Boinc Manager.
ID: 111513 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111520 - Posted: 8 Apr 2023, 15:59:27 UTC - in response to Message 111513.  

No, I don't want that any project will override parameters.

Then use local preferences set in each host's Boinc Manager.

And here we come back to my original problem: Asteroids@Home ignores the local preferences that I have set in the boinc manager.
ID: 111520 · Report as offensive
Profile Keith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 874
United States
Message 111524 - Posted: 8 Apr 2023, 19:23:30 UTC - in response to Message 111520.  

No, I don't want that any project will override parameters.

Then use local preferences set in each host's Boinc Manager.

And here we come back to my original problem: Asteroids@Home ignores the local preferences that I have set in the boinc manager.

Not sure what your problem is then. I have no issues with Asteroids obeying and following my local preferences set on each of my hosts.
ID: 111524 · Report as offensive
Magiceye04

Send message
Joined: 13 Aug 10
Posts: 24
Germany
Message 111544 - Posted: 12 Apr 2023, 21:34:54 UTC - in response to Message 111341.  
Last modified: 12 Apr 2023, 21:35:32 UTC

I have heard that it is a bit tricky to persuade Ubuntu to work together with a newer version. But if it is needed, I would try.
Or is there an alternative for the <max_concurrent> to be used in the old BM ?
You could try

https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/boinc

I use that on Linux Mint 20.3. It doesn't install on Mint 21.1, which I think is based on Ubuntu 20.4, but that may have different security policies. If you can load it, it's been reliable for me.

I have updated to the BM of this PPA - but without any change.
Setting is 0% for Asteroids and 0.1 days - I got 2 days work.
And WCG got 8 days of work.
Most will be aborted tomorrow by me as usual.
ID: 111544 · Report as offensive
1 · 2 · Next

Message boards : Questions and problems : Too many WUs were downloaded, all settings are ignored

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.