Is there a way to get BOINC to give priority to Tasks with short "time to deadline"?

Message boards : Questions and problems : Is there a way to get BOINC to give priority to Tasks with short "time to deadline"?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
SteveTheCynic

Send message
Joined: 30 Dec 21
Posts: 8
Message 107587 - Posted: 27 Mar 2022, 10:40:16 UTC
Last modified: 27 Mar 2022, 10:40:54 UTC

Or at least to show me (on the BOINC Manager display) why the longer-deadline tasks are running instead of the shorter ones?

I have a couple of locally-configured projects, and Science United for the rest. I tend to see some tasks with deadlines around two days from now sitting idle ("waiting to start" or "waiting to run"), with some that are due in seven to ten days' time that are running. Even when you include the estimated remaining run-time of the tasks, it seems perverse to leave soon-to-expire tasks idle while not-soon tasks run.

Windows 11, i9-10980XE (18 HT cores), NVidia 2080Ti, 64GB system RAM.

EDIT: BOINC 7.16.20
ID: 107587 · Report as offensive
Bryn Mawr
Help desk expert

Send message
Joined: 31 Dec 18
Posts: 284
United Kingdom
Message 107588 - Posted: 27 Mar 2022, 10:52:11 UTC - in response to Message 107587.  

Or at least to show me (on the BOINC Manager display) why the longer-deadline tasks are running instead of the shorter ones?

I have a couple of locally-configured projects, and Science United for the rest. I tend to see some tasks with deadlines around two days from now sitting idle ("waiting to start" or "waiting to run"), with some that are due in seven to ten days' time that are running. Even when you include the estimated remaining run-time of the tasks, it seems perverse to leave soon-to-expire tasks idle while not-soon tasks run.

Windows 11, i9-10980XE (18 HT cores), NVidia 2080Ti, 64GB system RAM.

EDIT: BOINC 7.16.20


If the short deadline tasks are in danger of not completing then Boinc will put them into high priority mode and drop any other tasks until they are completed. Until that point is reached Boinc will run the tasks according to the project share you have assigned.
ID: 107588 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 107589 - Posted: 27 Mar 2022, 11:51:04 UTC - in response to Message 107587.  

If BOINC were to exclusively run the tasks with the short deadline first, then when would it run the tasks with the long deadline?
BOINC makes calculations based on the information that the project gives to the tasks whether it can run tasks within the given deadline, and run them accordingly. And even if it cannot run all tasks on your system within the deadline, there's no man overboard as most projects have a redundancy where tasks are run by multiple computers. And tasks not run on yours will be run on another's.
ID: 107589 · Report as offensive
SteveTheCynic

Send message
Joined: 30 Dec 21
Posts: 8
Message 107608 - Posted: 28 Mar 2022, 17:25:08 UTC - in response to Message 107588.  

If the short deadline tasks are in danger of not completing then Boinc will put them into high priority mode and drop any other tasks until they are completed. Until that point is reached Boinc will run the tasks according to the project share you have assigned.

OK. That's not remotely clear from the UI, and I haven't touched the project share, which means, I presume, that it's equal everywhere.

And the observed behaviour isn't just between different projects, but between different tasks of the same project.
ID: 107608 · Report as offensive
SteveTheCynic

Send message
Joined: 30 Dec 21
Posts: 8
Message 107609 - Posted: 28 Mar 2022, 17:32:30 UTC - in response to Message 107589.  

If BOINC were to exclusively run the tasks with the short deadline first, then when would it run the tasks with the long deadline?

Is that a trick question? The obvious answer is that it would start to pick them up as they become (progressively) the shortest deadlines.
BOINC makes calculations based on the information that the project gives to the tasks whether it can run tasks within the given deadline, and run them accordingly. And even if it cannot run all tasks on your system within the deadline, there's no man overboard as most projects have a redundancy where tasks are run by multiple computers. And tasks not run on yours will be run on another's.

I know that, but the question is why it seems to reject the idea of giving priority to running the short-deadline ones. It's pretty close to 100% consistent. Short-deadline (but still achievable) tasks sit idle while long-deadline (even taking into account the estimated time to completion) tasks run. Even on the same project. When the difference in estimated completion time is several days or even upwards of a week. Even when the short-deadline task would take two minutes to run from zero to done.

If there's some other priority (like what was done for Rosetta when they started doing Covid folding at the beginning of the pandemic), that's fine, but a way to show it on the UI would help.
ID: 107609 · Report as offensive
Bryn Mawr
Help desk expert

Send message
Joined: 31 Dec 18
Posts: 284
United Kingdom
Message 107610 - Posted: 28 Mar 2022, 17:40:57 UTC - in response to Message 107609.  


Is that a trick question? The obvious answer is that it would start to pick them up as they become (progressively) the shortest deadlines.


As an example, CDPN has a deadline of one year but only take about 10 days to run on most modern systems. It would make no sense to delay them by 350 days before running them.
ID: 107610 · Report as offensive
SteveTheCynic

Send message
Joined: 30 Dec 21
Posts: 8
Message 107622 - Posted: 29 Mar 2022, 5:17:37 UTC - in response to Message 107610.  

As an example, CDPN has a deadline of one year but only take about 10 days to run on most modern systems. It would make no sense to delay them by 350 days before running them.


OK, Fair enough. I didn't realise anything would put out deadlines *that* long. The projects I've been running (i.e. what Science United drops on me) generally have deadlines up to about ten days, with three or four hour runtimes at the longest.
ID: 107622 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 107624 - Posted: 29 Mar 2022, 6:50:20 UTC - in response to Message 107622.  

OK, Fair enough. I didn't realise anything would put out deadlines *that* long. The projects I've been running (i.e. what Science United drops on me) generally have deadlines up to about ten days, with three or four hour runtimes at the longest.


If you want more control over how BOINC does things, drop SU and attach projects directly. The other issue with SU is that you don't get automagical access to the project forums.
ID: 107624 · Report as offensive
Nick Name

Send message
Joined: 14 Aug 19
Posts: 55
United States
Message 107699 - Posted: 3 Apr 2022, 20:48:58 UTC

I don't know if you have any control over Science United tasks, but you can force standard BOINC tasks to run by doing the following.

1) Set the project you want to to run now to NNT (No New Tasks).
2) Suspend the other projects or pause all other tasks. Click Show All Tasks in the manager to see every task in the queue.

This will start the project tasks you want. If you don't set that project to NNT you may find it downloads more work for that project. That's counterproductive so make sure to set to NNT first.

You should set a low cache to keep this from happening. If you're using local settings in the manager, go to Options -> Computing Preferences -> Other = Store At Least. Normally I don't like to use more than a day. Things should balance out over time, think weeks rather than days, but you can help things along by setting a minimal cache.

CPDN is an outlier, it's the only project with such extreme deadlines and in my opinion shouldn't be run with other projects for that reason. You will not have any control over that with SU, and I don't recommend running SU either.
Team USA forum
Follow us on Twitter
Help us #crunchforcures!
ID: 107699 · Report as offensive
Jean-David

Send message
Joined: 19 Dec 05
Posts: 89
United States
Message 107785 - Posted: 10 Apr 2022, 17:00:53 UTC - in response to Message 107682.  

Pore example, only CPDN set absurd deadline


CPDN deadlines were not absurd. They may be a little long these days with faster machines, and shorter work-units, so perhaps they could be cut in half, or even to 25%. Mine currently take 4 to 8 days, depending on what work units I get. But a new set of work units (OpenIFS -- not yet available) might take a long time. I am told they are RAM hogs.

In the past, there were many CPDN tasks (most of them) that took several months to complete on the machine I had. It had two Xeon processors that ran at 3.06 GHz, which was as fast as I could get at the time. It had 8GBytes of RAM that was considered a lot in those days.
ID: 107785 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 107792 - Posted: 11 Apr 2022, 7:12:41 UTC

Anyway they haven't had Windows work for a year or 3, so they're off my list.
December 2020 was the last Windows batch. Just over a week ago there was some Windows work in the testing branch which will lead to some main site work once they have confirmed that the changes they made to some files really are the reason the last batch succeeded and earlier ones didn't.
ID: 107792 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 107794 - Posted: 11 Apr 2022, 8:17:05 UTC

Thanks for letting me know, I'll reconnect and see if anything turns up. I take it the testing branch isn't customer facing? Or can I select beta work in the options?


I had been a regular contributor to the forums on CPDN for over 12 years before I was invited to join the testing branch. Currently those who are part of the testing branch are the moderators on the CPDN forum of which I am one and some of those working for the project in Oxford. There are enough of us that most batches only a few of us get any work. (Batches are usually between three and ten tasks with somewhere in the region of 50 computers between us but only ten computers having any recent credit. Certainly not one to even think about if you care about credit as quite a lot of tasks fail and don't grant any credit. If you want regular work from CPDN best bet is to use WSL or another virtualisation such as VB to then run Linux tasks as well.
ID: 107794 · Report as offensive
computezrmle

Send message
Joined: 2 Feb 22
Posts: 81
Germany
Message 107797 - Posted: 11 Apr 2022, 9:22:30 UTC - in response to Message 107796.  

... Is it easy enough to set up to grab CPDN Linux tasks?

Follow the VirtualBox manual to create a new VM:
https://www.virtualbox.org/manual/ch01.html#gui-createvm

Then install a Linux distribution as guest.
Then install BOINC on that Linux guest and attach to a project.

Most native projects (like CPDN) should run without problems.
Vbox projects inside the VM may or may not run depending on the CPU, the parameters used during VM creation and the guest Linux.

Be aware that you need to manually pause/resume that VM when you shutdown/restart the computer.
The BOINC client on your host doesn't do this as it doesn't even know that it exists.
ID: 107797 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 107799 - Posted: 11 Apr 2022, 9:40:52 UTC - in response to Message 107796.  

I wasn't aware I could use VB to get Linux tasks. I use VB on projects that use it like LHC, Rosetta, Cosmology. Is it easy enough to set up to grab CPDN Linux tasks? I have a fair grasp of computers, but I'm not a programmer, and I'm only familiar with Windows.


If using VB you would then need to install Linux including the 32bit libraries that some projects including CPDN require in VB. If running a recent Windows version (10 or 11) - not sure about earlier WSL is the easiest way to do it according to those I know iwth Windows boxes. Having defenestrated over 20 years ago I use WINE from within Linux when I need to run Windows tasks.
ID: 107799 · Report as offensive
1 · 2 · 3 · Next

Message boards : Questions and problems : Is there a way to get BOINC to give priority to Tasks with short "time to deadline"?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.