app_config or other means to limit downloaded WU?

Message boards : Questions and problems : app_config or other means to limit downloaded WU?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 65407 - Posted: 13 Nov 2015, 17:25:27 UTC

I have a 6 core 8GB machine that can handle 2 ATLAS, 2 lVHC and 2 of some other project within the 8GB without regular core throttle-down from page swapping.

I used APP_CONFIG.XML to limit WU's concurrents.

The problem is some other project d/ling a days worth of work for 6 cores when it's only allotted 2 cores for the day in it's APP_CONFIG and then ATLAS, which has the highest resource share, refuses to d/l new WU because it sees the CPU cache as full. So I have to go and abort a bunch of lower resource share WU to make room for the new ATLAS work.

I want to limit the WU d/l of the other projects because I'm micro managing the work load now.

Sure the easiest solution would be more RAM but that's not happening and this work load is optimal for the machine's configuration. There's got to be some way to better manage WU downloads per project.
ID: 65407 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15484
Netherlands
Message 65409 - Posted: 13 Nov 2015, 17:46:19 UTC - in response to Message 65407.  

You could run ATLAS separately in a BOINC running in a VirtualBox VM and tell it to use 2 CPU cores. Then you tell the BOINC outside the VM to use max 4 cores.

You can't use app_config.xml to set up the amount of work you want to download and cache as that's not an application decision. You'll have to tell BOINC to download less work. So set up a low value for Store at least N days of work (like zero), and an equal low value for Store up to an additional N days of work. Or tell BOINC to not use all CPU cores.

There's got to be some way to better manage WU downloads per project.

The easiest thing is not to try to run 3 projects with tasks with wildly varying run times on the same computer.
ID: 65409 · Report as offensive
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 65410 - Posted: 13 Nov 2015, 18:14:36 UTC - in response to Message 65409.  
Last modified: 13 Nov 2015, 18:47:26 UTC

You could run ATLAS separately in a BOINC running in a VirtualBox VM and tell it to use 2 CPU cores. Then you tell the BOINC outside the VM to use max 4 cores.


Never tried this. I'm going through the new VM creation wizard now.


You can't use app_config.xml to set up the amount of work you want to download and cache as that's not an application decision. You'll have to tell BOINC to download less work. So set up a low value for Store at least N days of work (like zero), and an equal low value for Store up to an additional N days of work. Or tell BOINC to not use all CPU cores.


It's currently set up for 1 day and 0 additional. I'll try and see what effect 0/0 has on the d/l. That should be a just-in-time configuration.


There's got to be some way to better manage WU downloads per project.

The easiest thing is not to try to run 3 projects with tasks with wildly varying run times on the same computer.


I was thinking more like BOINC reading the APP_CONFIG concurrent value and using it for WU d/l management.
If I could run 6 ATLAS on that machine, I would, but there's no money for more RAM this winter and so lVHC gets it's cut. lVHC is limited to 2 WU by the project, so other projects must be run on the machine if you want full load.

Thanks for the quick response.
ID: 65410 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15484
Netherlands
Message 65411 - Posted: 13 Nov 2015, 18:34:57 UTC - in response to Message 65410.  

Never tried this. So I need to d/l or create an ISO OS which runs BOINC; something like Puppy Linux? You know of a repository with some pre-made ISO/VDI's running BOINC?

Theoretically, what you do is run VBox, install an operating system in it and install BOINC on that, then add the project to that BOINC. So you could go for any flavor of Linux, and install either BOINC from repository or from Berkeley.

When setting up this VM before you install the OS, you can set it to use a max of 2 CPU cores. That way the BOINC in the VM will also only use 2 cores. You do still have to set the BOINC outside the VM to use only 4 cores, or else you'll get a severely overloaded system.

I'll try and see what effect 0/0 has on the d/l. That should be a just-in-time configuration.

0 and 0 will download 1 task for each available core, unless the work is so fast, then it might download a little more.

I was thinking more like BOINC reading the APP_CONFIG concurrent value and using it for WU d/l management.

The max_concurrent is only used to tell BOINC to run this many tasks at maximum for this application, not how many BOINC can cache of them.

If I could run 6 ATLAS on that machine, I would, but there's no money for more RAM this winter and so lVHC gets it's cut. lVHC is limited to 2 WU by the project, so other projects must be run on the machine if you want full load.

Well, you could try running CPDN on a low cache setting. Those weather models take quite a bit of time.
ID: 65411 · Report as offensive
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 65413 - Posted: 13 Nov 2015, 18:59:00 UTC - in response to Message 65411.  


Theoretically, what you do is run VBox, install an operating system in it and install BOINC on that, then add the project to that BOINC. So you could go for any flavor of Linux, and install either BOINC from repository or from Berkeley.

When setting up this VM before you install the OS, you can set it to use a max of 2 CPU cores. That way the BOINC in the VM will also only use 2 cores. You do still have to set the BOINC outside the VM to use only 4 cores, or else you'll get a severely overloaded system.


Going through the VBox creation wizard now and will push through the learning curve. I think the 2 core VM for this BOINC should be for the extra WU's instead of trying to put the ATLAS or vLHC VM's within another VM. That sounds like asking for headaches especially since both those project love to gobble up disk space.

If I could run 6 ATLAS on that machine, I would, but there's no money for more RAM this winter and so lVHC gets it's cut. lVHC is limited to 2 WU by the project, so other projects must be run on the machine if you want full load.

Well, you could try running CPDN on a low cache setting. Those weather models take quite a bit of time.


Climate Prediction have had no Windows WU's for 2 weeks and so I'll have to install Linux on 5 boxes and been stalling on that project. I'm going to install Zorin 10 on my daily laptop and try to run all my daily tasks on Linux now and not move onto Windows X. I've been preparing for this for a while by moving to almost all PortableApps.com programs for most needs and those should all work under Wine.

Thanks for the solutions.
ID: 65413 · Report as offensive
Les Bayliss
Help desk expert

Send message
Joined: 25 Nov 05
Posts: 1654
Australia
Message 65417 - Posted: 13 Nov 2015, 19:30:17 UTC - in response to Message 65413.  

There's a large batch of Windows EU tasks being loaded as of the time of this post. But there are thousands of Windows computers hanging around ready to pounce.
ID: 65417 · Report as offensive
Les Bayliss
Help desk expert

Send message
Joined: 25 Nov 05
Posts: 1654
Australia
Message 65422 - Posted: 13 Nov 2015, 20:19:23 UTC - in response to Message 65417.  

And it seems that there is a problem with some of the many files on the latest EU batch, so you'll need to keep an eye on the cpdn Message boards about a fix.
ID: 65422 · Report as offensive
Les Bayliss
Help desk expert

Send message
Joined: 25 Nov 05
Posts: 1654
Australia
Message 65440 - Posted: 15 Nov 2015, 0:59:08 UTC

The cpdn problem has been sorted, and new tasks are being created.
They appear to be short (~37 hours), so the batch of 6-7 thousand won't last long.
ID: 65440 · Report as offensive

Message boards : Questions and problems : app_config or other means to limit downloaded WU?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.