Scheduling priority (question and wish list)

Message boards : BOINC client : Scheduling priority (question and wish list)
Message board moderation

To post messages, you must log in.

AuthorMessage
Werinbert

Send message
Joined: 9 Jan 13
Posts: 28
United States
Message 47152 - Posted: 9 Jan 2013, 23:52:28 UTC

I am running BOINC 7.0.28

Clicking to BOINC manager > Project tab > Properties command

Question:
First line under Scheduling it says "scheduling priority" what does this value actually represent. It always seems negative (except when a project has no work and is suspended, in which case it is 0). Assuming that this is the priority in identifying which projects get run, the values do not seem to correlate to the projects being proccessed (i.e. sometimes the project is a greater negative, sometimes not). I compared GPU vs GPU and CPU vs CPU projects (I am not running any individual project on both CPU and GPU on the same computer).

Wish list:
Is it possible to add the long-term and short-term debts to this command screen to:
1) make it easier to tweek my machines to better match my desired overall project balancing (especially over multiple machines), and
2) present an easier to read metric that new users can look up and see that the computer is in fact trying to balance the projects. For now new users have to take our word for it that the computer is in fact balancing and will be balanced in the long run.
ID: 47152 · Report as offensive
kdsjsdj

Send message
Joined: 5 Jan 13
Posts: 81
Message 47154 - Posted: 10 Jan 2013, 3:49:48 UTC - in response to Message 47152.  

I think short and long term debts are not used anymore?

I've been working on something that might be even better. BOINC client keeps a record of every job it completes. The records are kept in files that are named according to the URL of the project. Look in your BOINC data directory and you will find files that start with "job_log_" followed by a project URL. For example if you crunch the abc@home project you'll have a file named job_log_abcathome.com.txt.

Open one of those files in your text editor. Do not save the file ever. Note it consists of lines such as:

1357259323 ue 2419.444781 ct 13938.880000 fe 7200000000000 nm abc_sieve_wu_09314348_0 et 14195.942899

The red field is a Unix date-time stamp. It tells us when the result was uploaded to the project server. The number is easily converted to a day/month/year hh:mm:ss format
The blue field is the number of seconds the CPU worked on the task.
The green field is the elapsed time for the task.

I have a Python script that opens each of the job_log_ files it finds in your BOINC data directory and adds up all the elapsed times and gives you the percentage of the total that each of your projects received. Would you be interested in something like that?

We can go beyond that. With additional code added to it the script could keep running logs of the percentages and call graphing software that creates a line graph showing shares received per project over time so you can observe trends. It could produce a pie chart that shows the actual share each of your projects received as of Jan-5-2012 12:00:00, for example.

The raw data is there, in the job_log_*.txt files. We can do whatever we want with it.

To get the basic features I've described you would need to install the Python interpreter. Python runs on several different platforms including Linux, OSX and Windows. For the graphing features you would need to install gnuplot, the graphing-charting package I am most familiar with and partial to at this time. If you have some other preference then I might be able to accommodate it but no guarantees. For example the script could output data in CSV files you could feed to your spreadsheet program, some of those have somewhat sophisticated graphs and charts.
ID: 47154 · Report as offensive
Werinbert

Send message
Joined: 9 Jan 13
Posts: 28
United States
Message 47157 - Posted: 10 Jan 2013, 11:42:16 UTC - in response to Message 47154.  

Thanks, unfortunately that is not what I am looking for. I am hoping to get a better grasp on how BOINC comes up with its scheduling. Which is forward looking rather than backward looking as your script would do. I am also hoping that there would be some quick and easy metric shown in the Manager that would give me a rough idea what the next task or next couple of tasks that would be run.

Most of the FAQ, info pages, and threads are about the older version of BOINC (6) and not about the curent one (7). This is probably why I thought that there might still be the short/long-term debts. Again does anyone know what the value of the sheduling priority represents?

And thanks for the insight to the job_log files.
ID: 47157 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 47160 - Posted: 10 Jan 2013, 12:25:06 UTC - in response to Message 47157.  
Last modified: 10 Jan 2013, 12:34:57 UTC

The inter project debt-system is now credit/points based and prioritized to do a single project bulk download **, process the work and move on to the next project... in short. The client v7 is not/no longer trying to constantly balance from hour to hour/day to day, rather does this over a longer period. More efficient, less pre-empting [less tasks in memory with "waiting to run" if LAIM is on, less lost time reverting to last checkpoints if LAIM is off, though the preempting normally happens these days at checkpoint saves], and less average age of buffered tasks to name a few things.

There's actually a page for that to explain http://boinc.berkeley.edu/trac/wiki/CreditNew and since there are so many versions floating around in production, old and new version explanations need to stay around [At times if something new is implemented, the old ways / functions / options are removed... don't know why]. Many volunteers don't upgrade long as things keep working to their needs.

** Only the project with the highest [positive] scheduling priority will get a work fetch permission. Once the Minimum work buffer drops below setting, will a new work fetch be issued for the next project with highest scheduling priority... sort off.
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 47160 · Report as offensive
kdsjsdj

Send message
Joined: 5 Jan 13
Posts: 81
Message 47162 - Posted: 10 Jan 2013, 13:23:12 UTC - in response to Message 47157.  

Now I see. You are interested in what the client is about to do or what it thinks it should do; my script would (over the long run) tell you whether it actually does what it's supposed to do which is honor the project shares you set.

ID: 47162 · Report as offensive
Werinbert

Send message
Joined: 9 Jan 13
Posts: 28
United States
Message 47163 - Posted: 10 Jan 2013, 20:37:22 UTC - in response to Message 47160.  

The inter project debt-system is now credit/points based and prioritized to do a single project bulk download **, process the work and move on to the next project...


Sort explains the current semi-fiasco on my secondary computer. I was running two projects at 300 and 50 weighting respectively, and added a third at 100. This third project downloaded a single task and processed it then the manager proceeded to download 20 more tasks; a whopping 10 days of work when I only had my cache/buffers set for 2 days min and 0.1 days extra. This is on top of all the work that I still had accrued for my other two projects. (This computer is also the one that is not running 24/7 so we'll see if I can get the results out in a timely manner without aborting/over-running the deadline.)
ID: 47163 · Report as offensive
kdsjsdj

Send message
Joined: 5 Jan 13
Posts: 81
Message 47164 - Posted: 10 Jan 2013, 22:47:58 UTC - in response to Message 47163.  

That's very common and precisely the reason I never set cache higher than 0.1 and 0. Oh I *know* it's not supposed to happen but it does. You can chase the reason why it happens forever. You might find a clear answer you might not. Eventually you stop banging your head on the wall and just use a very small cache, especially when you add a new project as that seems to be when it happens. (Reduce the cache, add the new project, allow the numbers to settle then slowly increase your cache).

Or you abort lots of tasks which is not the end of the world anyway. You abort them, they get resent, no biggee.

Or you let the client abort them when it sees the deadline is passed and the task hasn't started yet. That works not too bad, sometimes.
ID: 47164 · Report as offensive

Message boards : BOINC client : Scheduling priority (question and wish list)

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.