Project-by-project network disable (similar to communications deferred)

This has also been suggested on a per file basis.

I agree with Mike. When the cpdn servers could not accept intermediate or final file uploads for a recent prolonged period, some multi-project crunchers who followed our advice to suspend boinc network activity while the problem lasted, ran out of work on other projects. I think this problem arose whether members had dialup or broadband.

Mo

probably not worth the trouble

Ironic that this was closed on the same day that the CPDN administrator wrote:

"cpdn-upload1.comlab is shown as down at the moment. The server is running but it's shut down apache as the data partition is full. I've got nowhere else to put the data at the moment so this may well cause a problem for hadam3p uploads until I can obtain more hardware."

I think such a facility would be worth the trouble and not just for members of CPDN. The disk of the CPDN server that takes file uploads has filled up more than once and I would be surprised if this has never occurred on any other project. File upload servers of course also occasionally crash or fail.

When this has happened we have advised members to either suspend BOINC network activity altogether or suspend tasks before they complete. Not all members read this advice and some do not realise in time. They can find themselves with partially-uploaded files which I consider to be in a fragile state. Such files can be the end result of long periods of processing.

If members in this position urgently need to fetch work to keep the computer busy, the only possibilities at the moment are to either suspend network activity altogether letting at least part of the computer's processing capability run idle, or to allow BOINC network activity and put partially-uploaded files at greater risk.

Now that CUDA devices and multi-core computers are increasingly common, requiring large amounts of work to keep all cores busy, I consider MikeMarsUK's proposal even more desirable than it was two years ago.

I am therefore taking the liberty of reopening this ticket in the hope that Mike's proposal will be reconsidered.

I'm not sure what you mean by "fragile state". BOINC has a mechanism for backing off and retrying file transfers. If this mechanism doesn't work we need to fix it. We can't rely on user actions to provide reliability.

I consider files waiting for transfer, whether already partially uploaded or not, to be in a fragile state for several reasons:

* the moment a transfer is attempted files are on a 14-day countdown to abandonment by BOINC. An extension of this period was requested on one of the BOINC mailing lists but did not meet with universal approval, it being thought that two weeks is plenty of time to obtain funding for a new server, order it, take delivery and install it

* many users with files already waiting for transfer give priority to fetching new work and leave network activity enabled. This causes multiple failed upload attempts of the untransferable files. Every failed attempt increases the likelihood of file corruption

* the greatest risk to waiting untransferable files is probably user action. Because the only apparent solution within the BOINC Manager Transfers tab is the Retry button, users may attempt this repeatedly. When this does not work users may resort to increasingly desperate attempts, for example repeatedly disallowing and reallowing BOINC network activity. (I have seen this action cause BOINC to abandon a file.)

It would therefore be helpful if users had a button within the Transfers tab to suspend tranfers either of selected files or to a specific project. It would allow users to take a safe precaution, in many cases avoiding all the risks I have outlined. Some users, having taken safe action, would be spared some worry, and I believe that the proportion of successfully uploaded results would increase.

Milo, the CPDN programmer who usually looks after the servers, said yesterday after reading this ticket 'It sounds like it would be very useful, particularly now.'

Users at SETI have now also realised how useful this would be, during their current server outage:

http://setiathome.berkeley.edu/forum_thread.php?id=54188&nowrap=true#908786

I have implemented the requested functionality, tested by a number of users over the past 2 months.

The changes allow networking to be suspended and resumed for selected projects, adding a new "Suspend network"/"Resume network" button to BOINC Manager's Projects tab.

When project networking is suspended any in progress uploads will have their timers reset and upload will not be restarted until project networking is resumed. No scheduler requests will be made but any pending downloads for the project will be completed. The project's status will be displayed as "Network activity suspended by user".

If a network suspended project generates a trickle-up this will be shown in the project's status message as "Network activity suspended by user, Trickle upload pending".

A scheduler request can be forced at any time by clicking the Update button. That will send any pending trickle-up messages and (if required) request new work for the project. If new tasks are allocated any required downloads will be made automatically without the need to enable project networking.

The status message on the Tasks tab for completed tasks which haven't been uploaded will be "Uploading, project networking suspended".

The status message on the Transfers tab for blocked uploads will be "Upload pending, project networking suspended".

When project networking is resumed any blocked uploads will be started.

I have patches (at revision 18840) available for boinc_core_release_6_6a, boinc_core_release_6_8 and boinc_trunk but the attachment option seems to be disabled at the moment.

Thank you, Thyme, and also to your testers.

I assume that for the time being this is only available in your own private build.

Patch for boinc_trunk (revision 18844)

Patch for boinc_core_release_6_6a (revision 18844)

Patch for boinc_core_release_6_8 (revision 18844)

Replying to mo.v:

* many users with files already waiting for transfer give priority to fetching new work and leave network activity enabled. This causes multiple failed upload attempts of the untransferable files. Every failed attempt increases the likelihood of file corruption

How could a file get corrupted by just trying to upload it too many times?

* the greatest risk to waiting untransferable files is probably user action. Because the only apparent solution within the BOINC Manager Transfers tab is the Retry button, users may attempt this repeatedly. When this does not work users may resort to increasingly desperate attempts, for example repeatedly disallowing and reallowing BOINC network activity. (I have seen this action cause BOINC to abandon a file.)

BOINC abandons files if they spend more than 14 days trying to upload, but there is no limit for the number of retries (I thought there was, but checked code and there isn't).