Feature suggestion - sharing of partially completed WUs

Message boards : BOINC client : Feature suggestion - sharing of partially completed WUs
Message board moderation

To post messages, you must log in.

Profile A Starful of Moons

Send message
Joined: 2 Jun 16
Posts: 2
Message 76234 - Posted: 3 Mar 2017, 21:53:20 UTC

Hi there.

A few days ago I came up with an idea for a new feature which I think could do something about late workunit results being sent back to projects' servers.
Now, I don't know that well how the process of processing workunits works, and what is possible and what isn't, but hear me out.

My idea is: uploading back partially completed workunits, to be redistributed to other users to finish calculations. I think this could be a good feature that would do something about workunits not being finished before their deadlines. This could be good in the case of users who irregularly use their computers which leads to long overdue workunits still on their machines.

From an end user's perspective this could be implemented by putting two a new checkbox in the BOINC manager's options>computing settings window. The checkbox could say something like "upload unfinished results on exit" or something that gets the message across easily. Checking this box could instruct BOINC to send results from, say, the latest checkpoint of a workunit, back to the servers upon client shutdown on the local machine. The servers storing workunits could send the checkpointed data and workunit to a new computer to continue calculations, minimizing the number of late workunit results. Example: User A only processes 75% of the WU, and does not regularly use the computer. A's WU could be sent to user B's computer, which would compute the remaining 25%.

From the score system point of view: One of my ideas for this feature is giving partial credit to the user who started processing, and partial to the one who finished processing. Say, 50-50, or some other ratio depending on the amount of work each user has put into processing a single workunit and how easy or meaningful it would be to implement a new credit system like this.

But first, is this possible? I don't know much about how workunits themselves are built. Do they need to be fully processed on the same computer, or could they be transferred to a different computer halfway through completion? If yes, would the new computer have to have the same hardware config (cpu clock speed, ram, etc.) or would slight differences not matter? (slightly higher/lower cpu clock speed, higher/lower amount of ram on the new machine?) I'm at least sure that assuming they would have to be run on the same OS through the whole process is not mistaken of me.

If I am right, this will require altering the software on the servers and on the boinc client itself to enable them to recognize partially completed WUs and manage them appropriately. Would this be too big of a problem or..?

Now, I am not sure how big of a deal overdue workunits are, but if they are I hope this would be a good way to counter-act the occurrence. (or has this idea crossed someone else's mind before and they ended up saying "meh.."(I think I know that if a WU is not reported back in time, it is sent out again to be processed by another machine.)

I hope this idea will get the attention of someone from the development team, and that maybe it will get implemented if it is worth it.

TL;DR: Send partially completed work to another computer which finishes processing the WU.

Cheers, BOINCers.
The wasted computing time
The wasted youth
The pretty lies
The ugly truth
ID: 76234 · Report as offensive
Volunteer tester
Help desk expert

Send message
Joined: 25 Aug 06
Posts: 1596
Message 76236 - Posted: 4 Mar 2017, 0:22:57 UTC - in response to Message 76234.  

BOINC already has the 'trickle' facility which some projects such as CPDN and WCG utilize, to pick up where others have left off. It's entirely up to projects to decide if their tasks produce output which is suitable to be picked up. Many aren't for a myriad of reasons.
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 76236 · Report as offensive
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 306
Message 76245 - Posted: 4 Mar 2017, 13:58:11 UTC - in response to Message 76234.  

TL;DR: Send partially completed work to another computer which finishes processing the WU.

This mechanism is already available to all projects but is not always technically possible. It highly depends on the scientific application. Sometimes the checkpoint is just too big in size to upload. Sometimes the checkpoint is architecture dependent (you would need the same CPU and OS to resume).

Regarding the issue of irregular use of computers that then have overdue tasks there are several solutions already. None of them is perfect but they try to minimize the problem. The problem is that we can't take a look into the future. In order to make the system perfect a computer would need to learn about usage patterns and then decide if it can reach the deadline or not (given that the progress reported by the app is accurate). This is easy if you turn on BOINC only on weekdays from 9am to 5pm but gets complicated for other usage scenarios. You as a user would need to tell BOINC for how long you are going to turn off the computer or for how long you want it to use the computer each time you start or shut down your computer.
ID: 76245 · Report as offensive

Message boards : BOINC client : Feature suggestion - sharing of partially completed WUs

Copyright © 2018 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.