Totally frustrated by the work fetch algorithm that could automate fetching single, temp WU when high priority projects have no work.

Message boards : Questions and problems : Totally frustrated by the work fetch algorithm that could automate fetching single, temp WU when high priority projects have no work.
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 66333 - Posted: 21 Dec 2015, 19:20:47 UTC

I'm not new to BOINC, been running it since 2005 and typically 6 machines at once since 2011.
I have a goal of putting out work to what ever science project needs my computers the most and after being involved with SETI/BOINC since 1999, I want some badges so my sig line can also be impressive like so many others.

So my app_config files for all CPU projects (on this Dell laptop, other machines are focused on particular projects) look like this:

<app_config>
  <project_max_concurrent>1</project_max_concurrent>
</app_config>


This laptop is the test bench and is attached to 30 projects of which currently about 15 have less than 10,000 credit and no badges. So this Dell with 4 cores has 18 CPU projects it is seeking work from and 1 core is sitting idle when I wake up this morning and no updates will get it a 4th WU. The only way to get a 4th job is to manually go through the list suspending projects which have no work available. This is time consuming and the client should easily be able to automate this process. The client shouldn't be holding open a core for high resource share projects without WU's. ***The client should go ahead and request a singles, temporary WU from another project lower on the resource priority list till WU's become available on the projects that are out of work, which can last for days and weeks.***

For example, DENIS has been revamping their website and servers for weeks and Citizen's Grid has no work but I can never know for sure WHEN they will come back on line and my computers need to be ready and accepting work. I can NOT suspend these projects but that is the only way to guarantee that the other projects get work.

The client is set to grab 1 day and another 1 day buffer. I tried Jord's suggesting of setting both to 0 and that doesn't help nor does a setting of 0.15 days. The high resource share projects with no work gum up the works and they need to remain high resource share so they get work when it becomes available.

Here's the work fetch diagnostic:

12/21/2015 12:49:02 PM |  | Starting BOINC client version 7.6.9 for windows_x86_64
12/21/2015 12:49:02 PM |  | log flags: file_xfer, sched_ops, task, work_fetch_debug
12/21/2015 12:49:02 PM |  | Libraries: libcurl/7.39.0 OpenSSL/1.0.2a zlib/1.2.8
12/21/2015 12:49:02 PM |  | Data directory: (redacted)Data
12/21/2015 12:49:02 PM |  | Running under account (redacted)
12/21/2015 12:49:03 PM |  | CUDA: NVIDIA GPU 0: Quadro FX 3700M (driver version 337.88, CUDA version 6.0, compute capability 1.1, 1024MB, 744MB available, 512 GFLOPS peak)
12/21/2015 12:49:03 PM |  | OpenCL: NVIDIA GPU 0: Quadro FX 3700M (driver version 337.88, device version OpenCL 1.0 CUDA, 1024MB, 744MB available, 512 GFLOPS peak)
12/21/2015 12:49:03 PM | SETI@home | Found app_info.xml; using anonymous platform
12/21/2015 12:49:03 PM |  | Host name: dell-m6500
12/21/2015 12:49:03 PM |  | Processor: 4 GenuineIntel Intel(R) Core(TM) i7 CPU       M 620  @ 2.67GHz [Family 6 Model 37 Stepping 2]
12/21/2015 12:49:03 PM |  | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx smx tm2 pbe
12/21/2015 12:49:03 PM |  | OS: Microsoft Windows 7: Ultimate x64 Edition, Service Pack 1, (06.01.7601.00)
12/21/2015 12:49:03 PM |  | Memory: 7.99 GB physical, 16.49 GB virtual
12/21/2015 12:49:03 PM |  | Disk: 71.18 GB total, 50.99 GB free
12/21/2015 12:49:03 PM |  | Local time is UTC -6 hours
12/21/2015 12:49:03 PM | Asteroids@home | Found app_config.xml
12/21/2015 12:49:03 PM | ATLAS@home | Found app_config.xml
12/21/2015 12:49:03 PM | Citizen Science Grid | Found app_config.xml
12/21/2015 12:49:03 PM | climateprediction.net | Found app_config.xml
12/21/2015 12:49:03 PM | Cosmology@Home | Found app_config.xml
12/21/2015 12:49:03 PM | DENIS@Home | Found app_config.xml
12/21/2015 12:49:03 PM | DistributedDataMining | Found app_config.xml
12/21/2015 12:49:03 PM | Enigma@Home | Found app_config.xml
12/21/2015 12:49:03 PM | Leiden Classical | Found app_config.xml
12/21/2015 12:49:03 PM | LHC@home 1.0 | Found app_config.xml
12/21/2015 12:49:03 PM | malariacontrol.net | Found app_config.xml
12/21/2015 12:49:03 PM | MindModeling@Beta | Found app_config.xml
12/21/2015 12:49:03 PM | NFS@Home | Found app_config.xml
12/21/2015 12:49:03 PM | NumberFields@home | Found app_config.xml
12/21/2015 12:49:03 PM | Poem@Home | Found app_config.xml
12/21/2015 12:49:03 PM | pogs | Found app_config.xml
12/21/2015 12:49:03 PM | PrimeGrid | Found app_config.xml
12/21/2015 12:49:03 PM | rosetta@home | Found app_config.xml
12/21/2015 12:49:03 PM | SAT@home | Found app_config.xml
12/21/2015 12:49:03 PM | SZTAKI Desktop Grid | Found app_config.xml
12/21/2015 12:49:03 PM | Universe@Home | Found app_config.xml
12/21/2015 12:49:03 PM | VirtualLHC@home | Found app_config.xml
12/21/2015 12:49:03 PM | World Community Grid | Found app_config.xml
12/21/2015 12:49:03 PM | yafu | Found app_config.xml
12/21/2015 12:49:03 PM | yoyo@home | Found app_config.xml
12/21/2015 12:49:03 PM |  | Config: use all coprocessors
12/21/2015 12:49:03 PM | Asteroids@home | URL http://asteroidsathome.net/boinc/; Computer ID 52516; resource share 2
12/21/2015 12:49:03 PM | ATLAS@home | URL http://atlasathome.cern.ch/; Computer ID 31133; resource share 23
12/21/2015 12:49:03 PM | Bitcoin Utopia | URL http://www.bitcoinutopia.net/bitcoinutopia/; Computer ID 6814; resource share 1
12/21/2015 12:49:03 PM | BURP | URL http://burp.renderfarming.net/; Computer ID 67745; resource share 0
12/21/2015 12:49:03 PM | Citizen Science Grid | URL http://csgrid.org/csg/; Computer ID 17233; resource share 99
12/21/2015 12:49:03 PM | climateprediction.net | URL http://climateprediction.net/; Computer ID 1347460; resource share 10
12/21/2015 12:49:03 PM | Collatz Conjecture | URL http://boinc.thesonntags.com/collatz/; Computer ID 155815; resource share 0
12/21/2015 12:49:03 PM | Cosmology@Home | URL http://www.cosmologyathome.org/; Computer ID 277246; resource share 7
12/21/2015 12:49:03 PM | DENIS@Home | URL http://denis.usj.es/denisathome/; Computer ID 62796; resource share 10
12/21/2015 12:49:03 PM | DistributedDataMining | URL http://www.distributeddatamining.org/DistributedDataMining/; Computer ID 18410; resource share 9
12/21/2015 12:49:03 PM | Einstein@Home | URL http://einstein.phys.uwm.edu/; Computer ID 11695776; resource share 1
12/21/2015 12:49:03 PM | Enigma@Home | URL http://www.enigmaathome.net/; Computer ID 171877; resource share 4
12/21/2015 12:49:03 PM | GPUGRID | URL http://www.gpugrid.net/; Computer ID 190132; resource share 0
12/21/2015 12:49:03 PM | Leiden Classical | URL http://boinc.gorlaeus.net/; Computer ID 136557; resource share 14
12/21/2015 12:49:03 PM | LHC@home 1.0 | URL http://lhcathomeclassic.cern.ch/sixtrack/; Computer ID 10380163; resource share 11
12/21/2015 12:49:03 PM | malariacontrol.net | URL http://www.malariacontrol.net/; Computer ID 1660861; resource share 10
12/21/2015 12:49:03 PM | Milkyway@Home | URL http://milkyway.cs.rpi.edu/milkyway/; Computer ID 642971; resource share 1
12/21/2015 12:49:03 PM | MindModeling@Beta | URL http://mindmodeling.org/; Computer ID 73562; resource share 14
12/21/2015 12:49:03 PM | Moo! Wrapper | URL http://moowrap.net/; Computer ID 34406; resource share 1
12/21/2015 12:49:03 PM | NFS@Home | URL http://escatter11.fullerton.edu/nfs/; Computer ID 2174044; resource share 8
12/21/2015 12:49:03 PM | NumberFields@home | URL http://numberfields.asu.edu/NumberFields/; Computer ID 26363; resource share 9
12/21/2015 12:49:03 PM | Poem@Home | URL http://boinc.fzk.de/poem/; Computer ID 1255804; resource share 1
12/21/2015 12:49:03 PM | pogs | URL http://pogs.theskynet.org/pogs/; Computer ID 644886; resource share 4
12/21/2015 12:49:03 PM | PrimeGrid | URL http://www.primegrid.com/; Computer ID 479835; resource share 1
12/21/2015 12:49:03 PM | rosetta@home | URL http://boinc.bakerlab.org/rosetta/; Computer ID 2397021; resource share 3
12/21/2015 12:49:03 PM | SAT@home | URL http://sat.isa.ru/pdsat/; Computer ID 34810; resource share 4
12/21/2015 12:49:03 PM | SETI@home | URL http://setiathome.berkeley.edu/; Computer ID 7095070; resource share 1
12/21/2015 12:49:03 PM | SZTAKI Desktop Grid | URL http://szdg.lpds.sztaki.hu/szdg/; Computer ID 363910; resource share 24
12/21/2015 12:49:03 PM | Universe@Home | URL http://universeathome.pl/universe/; Computer ID 12897; resource share 10
12/21/2015 12:49:03 PM | VirtualLHC@home | URL http://lhcathome2.cern.ch/vLHCathome/; Computer ID 86779; resource share 22
12/21/2015 12:49:03 PM | World Community Grid | URL http://www.worldcommunitygrid.org/; Computer ID 3436048; resource share 3
12/21/2015 12:49:03 PM | WUProp@Home | URL http://wuprop.boinc-af.org/; Computer ID 88779; resource share 1
12/21/2015 12:49:03 PM | yafu | URL http://yafu.myfirewall.org/yafu/; Computer ID 11567; resource share 3
12/21/2015 12:49:03 PM | yoyo@home | URL http://www.rechenkraft.net/yoyo/; Computer ID 245469; resource share 6
12/21/2015 12:49:03 PM | World Community Grid | General prefs: from World Community Grid (last modified 01-Dec-2015 21:44:38)
12/21/2015 12:49:03 PM | World Community Grid | Host location: none
12/21/2015 12:49:03 PM | World Community Grid | General prefs: using your defaults
12/21/2015 12:49:03 PM |  | Reading preferences override file
12/21/2015 12:49:03 PM |  | Preferences:
12/21/2015 12:49:03 PM |  | max memory usage when active: 4090.09MB
12/21/2015 12:49:03 PM |  | max memory usage when idle: 6953.16MB
12/21/2015 12:49:03 PM |  | max disk usage: 25.00GB
12/21/2015 12:49:03 PM |  | (to change preferences, visit a project web site or select Preferences in the Manager)
12/21/2015 12:49:03 PM |  | [work_fetch] Request work fetch: Prefs update
12/21/2015 12:49:03 PM |  | [work_fetch] Request work fetch: Startup
12/21/2015 12:49:03 PM |  | Not using a proxy
12/21/2015 12:49:03 PM |  | Suspending computation - user request
12/21/2015 12:50:06 PM |  | [work_fetch] ------- start work fetch state -------
12/21/2015 12:50:06 PM |  | [work_fetch] target work buffer: 86400.00 + 86400.00 sec
12/21/2015 12:50:06 PM |  | [work_fetch] --- project states ---
12/21/2015 12:50:06 PM | Asteroids@home | [work_fetch] REC 0.001 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | ATLAS@home | [work_fetch] REC 0.000 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | Bitcoin Utopia | [work_fetch] REC 32.349 prio -0.721 can request work
12/21/2015 12:50:06 PM | BURP | [work_fetch] REC 0.000 prio -1000.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | Citizen Science Grid | [work_fetch] REC 0.000 prio -0.000 can request work
12/21/2015 12:50:06 PM | climateprediction.net | [work_fetch] REC 56.287 prio -0.126 can request work
12/21/2015 12:50:06 PM | Collatz Conjecture | [work_fetch] REC 0.000 prio -1000.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | Cosmology@Home | [work_fetch] REC 226.544 prio -0.725 can request work
12/21/2015 12:50:06 PM | DENIS@Home | [work_fetch] REC 0.000 prio -0.000 can request work
12/21/2015 12:50:06 PM | DistributedDataMining | [work_fetch] REC 1.454 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | Einstein@Home | [work_fetch] REC 8034.926 prio -179.475 can request work
12/21/2015 12:50:06 PM | Enigma@Home | [work_fetch] REC 323.386 prio -0.950 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | GPUGRID | [work_fetch] REC 0.000 prio -1000.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | Leiden Classical | [work_fetch] REC 1.132 prio -0.002 can request work
12/21/2015 12:50:06 PM | LHC@home 1.0 | [work_fetch] REC 0.347 prio -0.001 can request work
12/21/2015 12:50:06 PM | malariacontrol.net | [work_fetch] REC 0.000 prio -0.000 can request work
12/21/2015 12:50:06 PM | Milkyway@Home | [work_fetch] REC 1.499 prio -0.033 can request work
12/21/2015 12:50:06 PM | MindModeling@Beta | [work_fetch] REC 53.324 prio -0.085 can request work
12/21/2015 12:50:06 PM | Moo! Wrapper | [work_fetch] REC 0.007 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | NFS@Home | [work_fetch] REC 0.675 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | NumberFields@home | [work_fetch] REC 13.785 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | Poem@Home | [work_fetch] REC 3.244 prio -0.072 can request work
12/21/2015 12:50:06 PM | pogs | [work_fetch] REC 13.020 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | PrimeGrid | [work_fetch] REC 0.001 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | rosetta@home | [work_fetch] REC 232.481 prio -1.728 can request work
12/21/2015 12:50:06 PM | SAT@home | [work_fetch] REC 129.117 prio -0.720 can request work
12/21/2015 12:50:06 PM | SETI@home | [work_fetch] REC 11281.151 prio -0.205 can't request work: suspended via Manager
12/21/2015 12:50:06 PM | SZTAKI Desktop Grid | [work_fetch] REC 2.183 prio -0.002 can request work
12/21/2015 12:50:06 PM | Universe@Home | [work_fetch] REC 5.464 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | VirtualLHC@home | [work_fetch] REC 0.000 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 12:50:06 PM | World Community Grid | [work_fetch] REC 5.980 prio -0.044 can request work
12/21/2015 12:50:06 PM | WUProp@Home | [work_fetch] REC 0.009 prio -0.000 can't request work: non CPU intensive
12/21/2015 12:50:06 PM | yafu | [work_fetch] REC 984.521 prio -7.319 can request work
12/21/2015 12:50:06 PM | yoyo@home | [work_fetch] REC 190.616 prio -0.769 can request work
12/21/2015 12:50:06 PM |  | [work_fetch] --- state for CPU ---
12/21/2015 12:50:06 PM |  | [work_fetch] shortfall 158096.07 nidle 0.00 saturated 116639.62 busy 0.00
12/21/2015 12:50:06 PM | Asteroids@home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | ATLAS@home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | Bitcoin Utopia | [work_fetch] share 0.005
12/21/2015 12:50:06 PM | BURP | [work_fetch] share 0.000 zero resource share
12/21/2015 12:50:06 PM | Citizen Science Grid | [work_fetch] share 0.448
12/21/2015 12:50:06 PM | climateprediction.net | [work_fetch] share 0.045
12/21/2015 12:50:06 PM | Collatz Conjecture | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | Cosmology@Home | [work_fetch] share 0.032
12/21/2015 12:50:06 PM | DENIS@Home | [work_fetch] share 0.045
12/21/2015 12:50:06 PM | DistributedDataMining | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | Einstein@Home | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | Enigma@Home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | GPUGRID | [work_fetch] share 0.000 zero resource share
12/21/2015 12:50:06 PM | Leiden Classical | [work_fetch] share 0.063
12/21/2015 12:50:06 PM | LHC@home 1.0 | [work_fetch] share 0.050
12/21/2015 12:50:06 PM | malariacontrol.net | [work_fetch] share 0.045
12/21/2015 12:50:06 PM | Milkyway@Home | [work_fetch] share 0.005
12/21/2015 12:50:06 PM | MindModeling@Beta | [work_fetch] share 0.063
12/21/2015 12:50:06 PM | Moo! Wrapper | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | NFS@Home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | NumberFields@home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | Poem@Home | [work_fetch] share 0.005
12/21/2015 12:50:06 PM | pogs | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | PrimeGrid | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | rosetta@home | [work_fetch] share 0.014
12/21/2015 12:50:06 PM | SAT@home | [work_fetch] share 0.018
12/21/2015 12:50:06 PM | SETI@home | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | SZTAKI Desktop Grid | [work_fetch] share 0.109
12/21/2015 12:50:06 PM | Universe@Home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | VirtualLHC@home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | World Community Grid | [work_fetch] share 0.014
12/21/2015 12:50:06 PM | yafu | [work_fetch] share 0.014
12/21/2015 12:50:06 PM | yoyo@home | [work_fetch] share 0.027
12/21/2015 12:50:06 PM |  | [work_fetch] --- state for NVIDIA GPU ---
12/21/2015 12:50:06 PM |  | [work_fetch] shortfall 0.00 nidle 0.00 saturated 173205.32 busy 0.00
12/21/2015 12:50:06 PM | Asteroids@home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | ATLAS@home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | Bitcoin Utopia | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | BURP | [work_fetch] share 0.000 zero resource share
12/21/2015 12:50:06 PM | Citizen Science Grid | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | climateprediction.net | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | Collatz Conjecture | [work_fetch] share 0.000 zero resource share
12/21/2015 12:50:06 PM | Cosmology@Home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | DENIS@Home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | DistributedDataMining | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | Einstein@Home | [work_fetch] share 0.016
12/21/2015 12:50:06 PM | Enigma@Home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | GPUGRID | [work_fetch] share 0.000 zero resource share
12/21/2015 12:50:06 PM | Leiden Classical | [work_fetch] share 0.219
12/21/2015 12:50:06 PM | LHC@home 1.0 | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | malariacontrol.net | [work_fetch] share 0.156
12/21/2015 12:50:06 PM | Milkyway@Home | [work_fetch] share 0.016
12/21/2015 12:50:06 PM | MindModeling@Beta | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | Moo! Wrapper | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 12:50:06 PM | NFS@Home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | NumberFields@home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | Poem@Home | [work_fetch] share 0.016
12/21/2015 12:50:06 PM | pogs | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | PrimeGrid | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | rosetta@home | [work_fetch] share 0.047
12/21/2015 12:50:06 PM | SAT@home | [work_fetch] share 0.063
12/21/2015 12:50:06 PM | SETI@home | [work_fetch] share 0.000
12/21/2015 12:50:06 PM | SZTAKI Desktop Grid | [work_fetch] share 0.375
12/21/2015 12:50:06 PM | Universe@Home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | VirtualLHC@home | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | World Community Grid | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | yafu | [work_fetch] share 0.000 no applications
12/21/2015 12:50:06 PM | yoyo@home | [work_fetch] share 0.094
12/21/2015 12:50:06 PM |  | [work_fetch] ------- end work fetch state -------
[b]12/21/2015 12:50:06 PM |  | [work_fetch] No project chosen for work fetch[/b]


I tried Peter's suggestion of turning off the GPU's with:
<cc_config>
<options>
<no_gpus>1</no_gpus>
</options>
</cc_config>

but the results didn't change:
12/21/2015 12:59:48 PM |  | [work_fetch] ------- end work fetch state -------
12/21/2015 12:59:48 PM |  | [work_fetch] No project chosen for work fetch


To finally get some work I had to suspend the top 7 resource shares and finally Milkyway@home got some work.

12/21/2015 1:13:25 PM |  | [work_fetch] Request work fetch: application exited
12/21/2015 1:13:28 PM |  | [work_fetch] ------- start work fetch state -------
12/21/2015 1:13:28 PM |  | [work_fetch] target work buffer: 120960.00 + 86400.00 sec
12/21/2015 1:13:28 PM |  | [work_fetch] --- project states ---
12/21/2015 1:13:28 PM | Asteroids@home | [work_fetch] REC 0.001 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | ATLAS@home | [work_fetch] REC 0.000 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | Bitcoin Utopia | [work_fetch] REC 31.981 prio -0.126 can request work
12/21/2015 1:13:28 PM | BURP | [work_fetch] REC 0.000 prio -1000.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | Citizen Science Grid | [work_fetch] REC 0.000 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | climateprediction.net | [work_fetch] REC 55.648 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | Collatz Conjecture | [work_fetch] REC 0.000 prio -1000.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | Cosmology@Home | [work_fetch] REC 229.260 prio -0.132 can request work
12/21/2015 1:13:28 PM | DENIS@Home | [work_fetch] REC 0.000 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | DistributedDataMining | [work_fetch] REC 1.438 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | Einstein@Home | [work_fetch] REC 8081.100 prio -32.148 can request work
12/21/2015 1:13:28 PM | Enigma@Home | [work_fetch] REC 325.002 prio -0.171 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | GPUGRID | [work_fetch] REC 0.000 prio -1000.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | Leiden Classical | [work_fetch] REC 1.119 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | LHC@home 1.0 | [work_fetch] REC 0.343 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | malariacontrol.net | [work_fetch] REC 0.000 prio -0.000 can request work
12/21/2015 1:13:28 PM | Milkyway@Home | [work_fetch] REC 2.529 prio -1.337 can't request work: scheduler RPC backoff (0.69 sec)
12/21/2015 1:13:28 PM | MindModeling@Beta | [work_fetch] REC 52.719 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | Moo! Wrapper | [work_fetch] REC 0.007 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | NFS@Home | [work_fetch] REC 0.667 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | NumberFields@home | [work_fetch] REC 13.629 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | Poem@Home | [work_fetch] REC 3.207 prio -0.013 can request work
12/21/2015 1:13:28 PM | pogs | [work_fetch] REC 12.872 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | PrimeGrid | [work_fetch] REC 0.001 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | rosetta@home | [work_fetch] REC 229.842 prio -0.303 can request work
12/21/2015 1:13:28 PM | SAT@home | [work_fetch] REC 127.651 prio -0.126 can request work
12/21/2015 1:13:28 PM | SETI@home | [work_fetch] REC 11153.087 prio -0.171 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | SZTAKI Desktop Grid | [work_fetch] REC 2.159 prio 0.000 can't request work: suspended via Manager
12/21/2015 1:13:28 PM | Universe@Home | [work_fetch] REC 5.402 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | VirtualLHC@home | [work_fetch] REC 0.000 prio -0.000 can't request work: "no new tasks" requested via Manager
12/21/2015 1:13:28 PM | World Community Grid | [work_fetch] REC 5.912 prio -0.008 can request work
12/21/2015 1:13:28 PM | WUProp@Home | [work_fetch] REC 0.009 prio -0.000 can't request work: non CPU intensive
12/21/2015 1:13:28 PM | yafu | [work_fetch] REC 973.345 prio -1.281 can request work
12/21/2015 1:13:28 PM | yoyo@home | [work_fetch] REC 193.740 prio -0.178 can request work
12/21/2015 1:13:28 PM |  | [work_fetch] --- state for CPU ---
12/21/2015 1:13:28 PM |  | [work_fetch] shortfall 203834.35 nidle 0.00 saturated 144940.06 busy 0.00
12/21/2015 1:13:28 PM | Asteroids@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | ATLAS@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Bitcoin Utopia | [work_fetch] share 0.036
12/21/2015 1:13:28 PM | BURP | [work_fetch] share 0.000 zero resource share
12/21/2015 1:13:28 PM | Citizen Science Grid | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | climateprediction.net | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Collatz Conjecture | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 1:13:28 PM | Cosmology@Home | [work_fetch] share 0.250
12/21/2015 1:13:28 PM | DENIS@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | DistributedDataMining | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Einstein@Home | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 1:13:28 PM | Enigma@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | GPUGRID | [work_fetch] share 0.000 zero resource share
12/21/2015 1:13:28 PM | Leiden Classical | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | LHC@home 1.0 | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | malariacontrol.net | [work_fetch] share 0.000 project is backed off  (resource backoff: 335.52, inc 600.00)
12/21/2015 1:13:28 PM | Milkyway@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | MindModeling@Beta | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Moo! Wrapper | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 1:13:28 PM | NFS@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | NumberFields@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Poem@Home | [work_fetch] share 0.036
12/21/2015 1:13:28 PM | pogs | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | PrimeGrid | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 1:13:28 PM | rosetta@home | [work_fetch] share 0.107
12/21/2015 1:13:28 PM | SAT@home | [work_fetch] share 0.143
12/21/2015 1:13:28 PM | SETI@home | [work_fetch] share 0.000 blocked by project preferences
12/21/2015 1:13:28 PM | SZTAKI Desktop Grid | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Universe@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | VirtualLHC@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | World Community Grid | [work_fetch] share 0.107
12/21/2015 1:13:28 PM | yafu | [work_fetch] share 0.107
12/21/2015 1:13:28 PM | yoyo@home | [work_fetch] share 0.214
12/21/2015 1:13:28 PM |  | [work_fetch] --- state for NVIDIA GPU ---
12/21/2015 1:13:28 PM |  | [work_fetch] shortfall 35317.99 nidle 0.00 saturated 172042.01 busy 0.00
12/21/2015 1:13:28 PM | Asteroids@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | ATLAS@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Bitcoin Utopia | [work_fetch] share 0.038
12/21/2015 1:13:28 PM | BURP | [work_fetch] share 0.000 zero resource share
12/21/2015 1:13:28 PM | Citizen Science Grid | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | climateprediction.net | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Collatz Conjecture | [work_fetch] share 0.000 zero resource share
12/21/2015 1:13:28 PM | Cosmology@Home | [work_fetch] share 0.269
12/21/2015 1:13:28 PM | DENIS@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | DistributedDataMining | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Einstein@Home | [work_fetch] share 0.038
12/21/2015 1:13:28 PM | Enigma@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | GPUGRID | [work_fetch] share 0.000 zero resource share
12/21/2015 1:13:28 PM | Leiden Classical | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | LHC@home 1.0 | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | malariacontrol.net | [work_fetch] share 0.000 project is backed off  (resource backoff: 339.82, inc 600.00)
12/21/2015 1:13:28 PM | Milkyway@Home | [work_fetch] share 0.000 project is backed off  (resource backoff: 398.59, inc 600.00)
12/21/2015 1:13:28 PM | MindModeling@Beta | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Moo! Wrapper | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | NFS@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | NumberFields@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Poem@Home | [work_fetch] share 0.038
12/21/2015 1:13:28 PM | pogs | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | PrimeGrid | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | rosetta@home | [work_fetch] share 0.115
12/21/2015 1:13:28 PM | SAT@home | [work_fetch] share 0.154
12/21/2015 1:13:28 PM | SETI@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | SZTAKI Desktop Grid | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | Universe@Home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | VirtualLHC@home | [work_fetch] share 0.000
12/21/2015 1:13:28 PM | World Community Grid | [work_fetch] share 0.115
12/21/2015 1:13:28 PM | yafu | [work_fetch] share 0.000 no applications
12/21/2015 1:13:28 PM | yoyo@home | [work_fetch] share 0.231
12/21/2015 1:13:28 PM |  | [work_fetch] ------- end work fetch state -------
12/21/2015 1:13:28 PM |  | [work_fetch] No project chosen for work fetch
12/21/2015 1:13:29 PM |  | [work_fetch] Request work fetch: Backoff ended for Milkyway@Home
12/21/2015 1:13:33 PM |  | [work_fetch] ------- start work fetch state -------
12/21/2015 1:13:33 PM |  | [work_fetch] target work buffer: 120960.00 + 86400.00 sec
12/21/2015 1:13:33 PM |  | [work_fetch] --- project states ---



So dealing with this lost me another hour of my life to a process that could easily be automated by the client and 7 of my high priority projects are now suspended for hours in which they might resume WU's.

It's VERY frustrating.
ID: 66333 · Report as offensive
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 66334 - Posted: 21 Dec 2015, 19:39:23 UTC - in response to Message 66333.  
Last modified: 21 Dec 2015, 19:45:48 UTC

Someone is going to say that my 1 WU per project is the complete fault here and I'll contend, like I have before, that the client should have options included in the BOINCmgr.exe for WU control and the fetch algorithm should take into account the app_config.xml <max_concurrent> settings that could be included in the client GUI management system.

There are several projects (vLHC, Atlas, BURP, others) that are limited by the server or by RAM restrictions so that they can not fill all cores of a machine with work.

Also, even with the best attempts at resource management, I've seen projects dominate the cores of machines and leave other projects in their dust. The resource management should include control over the cores available to each project, not BOINC as a whole.

Matthew Burch made some excellent points 2 years ago about resource allocation

If a local method of tracking resources is utilized, then it doesn't matter what individual projects do to try to skew the numbers, either intentionally, or by accident. In the example above, 100 hours of CPU is spent on one project for every 5 hours spent on another. Period. Done. No shenanigans allowed.
ID: 66334 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 66335 - Posted: 21 Dec 2015, 19:44:46 UTC - in response to Message 66333.  
Last modified: 21 Dec 2015, 19:46:40 UTC

There has been some talk about work fetch not handling max_concurrent well. I don't know if it has been fixed, I don't use it myself.

edit: Oh, never mind then.
ID: 66335 · Report as offensive
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 66385 - Posted: 24 Dec 2015, 12:13:27 UTC - in response to Message 66335.  

There has been some talk about work fetch not handling max_concurrent well. I don't know if it has been fixed, I don't use it myself.

edit: Oh, never mind then.


What was the "never mind" part about?

This is still a problem especially for projects like Atlas@home, vLHC@home, BURP, NFS large and potentially other future work units that could be increasingly using virtual machines with 1 to 2GB footprints, which can't have all 8 cores in most machines out there.

Three solutions are possible:

1) Make it easier to install multiple copies of BOINC with their own data directories and alot subsets of the CPU cores to each install. It's possible to start new BOINC.exe copies on new data directories and manage them from a new port assignment but the process is not friendly nor quick. It's much preferable to have a single BOINC manager with all running WU's seen at a glance.

2) Add core assignment control methods right into the BOINC manager left Project or Work Unit menus. Assigning preferred cores 1 - 8 (or 16 eventually) to individual projects would be handy.

3) Install a few BOINC clients into their own 2 core VM's. It would be inefficient or cause a crash to run project WU that are themselves VM's inside a BOINC dedicated VM so vBox projects should go to the main install of BOINC while others get the assigned cores of BOINC dedicated virtual machines.


I would so much prefer options 1 and 2 but will settle for 3 when I find the time to revamp the machines.

I hope the developers have plans to dedicate core assignment options built into the GUI as more computers come with 16 core and eventually 32 core CPUs and they also take into account projects with no work units for extended periods that stall work flow.

In order to get temp work units the client can go through the same heuristic that I have to follow.

1)suspend the highest resource share project.
2)call for global update (easiest is to change the buffer period by some small increment)
3) if no project is chosen for work download the suspend the next highest resource share project.
4)continue till some work units are received.
5)delete all but a few hours worth of new work units to keep the option for the high priority projects with no current work to get some work in a few hours later.
6) un-suspend projects and let work continue.
ID: 66385 · Report as offensive
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 66580 - Posted: 2 Jan 2016, 7:43:52 UTC

Setup the 6 core machine with two running clients and using the CC_CONFIG.XML options to adjust the number of cores that BOINC.EXE emulates to 6/2 cores. It's fetching work amazingly well now and the cores are at 100% usage all day long.

Anyone else having issues with work fetch from high resource projects with no work for days on end and RAM limited projects using APP_CONFIG.XML to limit <PROJECT_MAX_CONCURRENT>N</PROJECT_MAX_CONCURRENT>, these options in the CC_CONFIG.XML stored in the DATA directory (you'll likely have to create the file) can help sort out the work fetch flow.

<cc_config>
   <log_flags>
   </log_flags>
   <options>
       <allow_multiple_clients>1</allow_multiple_clients> - allows you to split into 2 or more clients.
       <fetch_minimal_work>1</fetch_minimal_work> - tells the client to fetch only 1 WU at a time and can stop some projects flooding the buffer while your client waits on a high resource project to get some work ready.
       <ncpus>X</ncpus> - where X is the number of cores per client you split into.  Actually, setting this higher than the cores in your machine has benefits.
       <fetch_on_update>1</fetch_on_update> - this causes projects that wouldn't normally get work because of the work-fetch algorithm decisions to fetch some WU's and goes well with <fetch_minimal_work>1</fetch_minimal_work>.

These keep your many machines running BOINC from flooding your single internet connection:
      <report_results_immediately>1</report_results_immediately>
      <max_file_xfers>2</max_file_xfers>

The rest of these are just some nice options.
      <skip_cpu_benchmarks>1</skip_cpu_benchmarks> - machines exclusively running BOINC 99.9% of the time don't need to be BMed.
      <start_delay>15</start_delay> - give you a chance to stop BOINC before apps start if you've made a mistake in configuration.
      <suppress_net_info>1</suppress_net_info> - doesn't report your IP.
     
    </options>
</cc_config>
ID: 66580 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5082
United Kingdom
Message 66583 - Posted: 2 Jan 2016, 9:06:22 UTC - in response to Message 66580.  

(you'll likely have to create the file)

The BOINC v7.6.xx range (you'll be wanting to use v7.6.22 now) have a handy 'Event log options...' picker under the Options menu, or Ctrl-Shift-F.

If you change something there, BOINC will write out a complete cc_config.xml file, nicely formatted, with everything in alphabetical order, and pre-populated with default values. It's much easier, and safer from typos, to start from that.

There's also a link in the picker direct to Client configuration, where the rest of the documentation lives.
ID: 66583 · Report as offensive
Profile marmot
Avatar

Send message
Joined: 16 Sep 13
Posts: 82
United States
Message 66601 - Posted: 4 Jan 2016, 18:30:37 UTC - in response to Message 66583.  

(you'll likely have to create the file)

The BOINC v7.6.xx range (you'll be wanting to use v7.6.22 now) have a handy 'Event log options...' picker under the Options menu, or Ctrl-Shift-F.

If you change something there, BOINC will write out a complete cc_config.xml file, nicely formatted, with everything in alphabetical order, and pre-populated with default values. It's much easier, and safer from typos, to start from that.


Nice! Will try it out.
ID: 66601 · Report as offensive

Message boards : Questions and problems : Totally frustrated by the work fetch algorithm that could automate fetching single, temp WU when high priority projects have no work.

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.