I do not understand the scheduler

Message boards : BOINC client : I do not understand the scheduler
Message board moderation

To post messages, you must log in.

AuthorMessage
Jean-David

Send message
Joined: 19 Dec 05
Posts: 89
United States
Message 9244 - Posted: 31 Mar 2007, 11:24:15 UTC

When my BOINC client requests more work, the request specifies the number of seconds of work desired. Recently it requested 1202 seconds of work from rosetta project and got a work unit requiring about 4 hours of time. Now that is not so bad, but when climate prediction requests a small amount of work, it gets an entire climate prediction work unit that normally takes over 6000 hours.

What is supposed to happen in such cases? I would have thought the BOINC server for the project would just say "No", but it seems it sends a work unit. After a bit of this, I have what seems to be too much work.

Are the project servers supposed to honor the work unit times or not?

ID: 9244 · Report as offensive
Pooh Bear 27

Send message
Joined: 15 Mar 07
Posts: 20
United States
Message 9245 - Posted: 31 Mar 2007, 11:51:34 UTC

First off, those numbers need to be more or less ignored by the project. You can not set your cache large enough for CPDN, so it would never download.

The small number means give me a minimal amount of work (1 unit?) to work on. If the number is larger and asking for 10 or 12 hours of work, then it will download that, and probably a little more. The number is just somewhat a guideline for the projects.

When you first sign up to a project, it sets the cache time at .1 day. A day has 24 hours, so 2.4 hours of work is the maximum it would ever ask for. I am unsure how many projects even have units that take much less than that. Since the project knows you need work, it will give you work.
ID: 9245 · Report as offensive
Jean-David

Send message
Joined: 19 Dec 05
Posts: 89
United States
Message 9248 - Posted: 31 Mar 2007, 12:50:40 UTC - in response to Message 9245.  

First off, those numbers need to be more or less ignored by the project. You can not set your cache large enough for CPDN, so it would never download.

The small number means give me a minimal amount of work (1 unit?) to work on. If the number is larger and asking for 10 or 12 hours of work, then it will download that, and probably a little more. The number is just somewhat a guideline for the projects.

When you first sign up to a project, it sets the cache time at .1 day. A day has 24 hours, so 2.4 hours of work is the maximum it would ever ask for. I am unsure how many projects even have units that take much less than that. Since the project knows you need work, it will give you work.


The trouble with this is that I am signed up for climate prediction, predictor, rosetta, setiathome, seasonal attribution project, and maleria control. My machine has two hyperthreaded processors, so it is treated as having 4 processors. That part seems OK.

But recently, seasonal attribution project and predictor were sending out no work, and both seasonal attribution project and climate prediction finished work units. So climate prediction sent a work unit. Then predictor finished a work unit and climate prediction sent another. Etc. Now I have 4 climate prediction work units to do and one seasonal attribution project. These are 5 biggies and I really do not think my client should be picking up any more work units. It asks for very tiny work units; e.g., 44 minutes, and gets bigger ones; e.g., 150 minutes. I really do not think it should have picked up the seasonal attribution one at all (over 1000 hours) when it already had 4 climate prediction ones. With four climate prediction work units, none near completion, it should really not seek any more work units, and if it insists on asking for them, should not the servers send out only ones equal or less in size than what is requested?
ID: 9248 · Report as offensive
Profile KSMarksPsych
Avatar

Send message
Joined: 30 Oct 05
Posts: 1239
United States
Message 9249 - Posted: 31 Mar 2007, 14:12:46 UTC
Last modified: 31 Mar 2007, 14:13:48 UTC

The CPDN servers can't send anything smaller because there is nothing smaller. CPDN models are all the same size. Huge.

Your best bet is to set CPDN to no new task and then abort the CPDN models you don't want. Once that is done, update CPDN to report those aborted models back to the server so they go back into the pool of work units to be sent out.
Kathryn :o)
ID: 9249 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1179
Argentina
Message 9251 - Posted: 31 Mar 2007, 16:38:02 UTC

I think the request is the minimum to send. We could say "server sends the amount of work requested, rounded up to the next full workunit".

As long as the client can handle it, that is. Remember those messages: "Not sending work: Won't finish in time".
ID: 9251 · Report as offensive
Aurora Borealis
Avatar

Send message
Joined: 8 Jan 06
Posts: 448
Canada
Message 9254 - Posted: 31 Mar 2007, 18:40:29 UTC
Last modified: 31 Mar 2007, 18:54:07 UTC

One thing that should be added is that the scheduler and work fetch policy is an extremely complex algorithm trying to negotiate conflicting needs and decisions. There are only a few people that fully understand its inner workings. I'm definitely not one of them.

It needs to handle projects with WU that take from a few minutes to thousands of hours to process with due dates from a few hours to over a year. Some projects supply WU that vary in processing times by a factor of 100 or more. At the same time Boinc is trying to make sure you don't run out of work but never miss a deadline. All this while trying to maintain the project resource share you assign to each project. Throw in the fact that some projects routinely supply work intermittently or simply go down, it's a wonder that it works as well as it does on single core systems.

All this being said, there are still kinks to be ironed out for multiprocessor systems. The added complications are still being worked on.

The suggestion about setting the Climate projects to 'No New Tasks' is a good one and I've been doing this on my C2D to make sure I don't accidentally download extra CPDN work before I'm ready to start crunching another one, which in my case wont be for 5 or 6 months, considering the number of projects I'm attached to. Also, the latest version of Boinc should be used as it is starting to address a few of the multiprocessor related scheduling problems.

Boinc V 7.4.36
Win7 i5 3.33G 4GB NVidia 470
ID: 9254 · Report as offensive
Jean-David

Send message
Joined: 19 Dec 05
Posts: 89
United States
Message 9263 - Posted: 1 Apr 2007, 1:03:52 UTC - in response to Message 9254.  

One thing that should be added is that the scheduler and work fetch policy is an extremely complex algorithm trying to negotiate conflicting needs and decisions. There are only a few people that fully understand its inner workings. I'm definitely not one of them.

It needs to handle projects with WU that take from a few minutes to thousands of hours to process with due dates from a few hours to over a year. Some projects supply WU that vary in processing times by a factor of 100 or more. At the same time Boinc is trying to make sure you don't run out of work but never miss a deadline. All this while trying to maintain the project resource share you assign to each project. Throw in the fact that some projects routinely supply work intermittently or simply go down, it's a wonder that it works as well as it does on single core systems.

All this being said, there are still kinks to be ironed out for multiprocessor systems. The added complications are still being worked on.

The suggestion about setting the Climate projects to 'No New Tasks' is a good one and I've been doing this on my C2D to make sure I don't accidentally download extra CPDN work before I'm ready to start crunching another one, which in my case wont be for 5 or 6 months, considering the number of projects I'm attached to. Also, the latest version of Boinc should be used as it is starting to address a few of the multiprocessor related scheduling problems.



I have now set the client to accept no new ClimatePrediction and no new SeasonalAttributionProject work units. However, the last three ClimatePrediction work units came pretty fast together so I would have to be pretty speedy on the button to keep it down to only one or two. I have also turned off downloading MalariaControl work units for a while, but will turn those on again in a week or so.
ID: 9263 · Report as offensive

Message boards : BOINC client : I do not understand the scheduler

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.