News on Project Outages

Message boards : Projects : News on Project Outages
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 10 · Next

AuthorMessage
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 36862 - Posted: 14 Feb 2011, 23:27:16 UTC - in response to Message 36816.  

ID: 36862 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 36863 - Posted: 14 Feb 2011, 23:29:09 UTC

Spinhenge is down -- perhaps one of the drive problems that seem to be making their rounds on BOINC projects these days.

The home page and web server are ok, but no uploads or downloads -- for 12 hours so far.
ID: 36863 · Report as offensive
Profile Gary Charpentier
Avatar

Send message
Joined: 23 Feb 08
Posts: 2462
United States
Message 36876 - Posted: 16 Feb 2011, 1:50:34 UTC

Seti Beta is down, but Seti Main is up. Strange.

ID: 36876 · Report as offensive
Claggy

Send message
Joined: 23 Apr 07
Posts: 1112
United Kingdom
Message 36877 - Posted: 16 Feb 2011, 2:05:27 UTC - in response to Message 36876.  

Seti Beta is down, but Seti Main is up. Strange.


Seti Beta is up, i've managed to report a few tasks tonight, but the Forums are down still,

Claggy

ID: 36877 · Report as offensive
Bernd

Send message
Joined: 24 Aug 09
Posts: 91
United States
Message 36885 - Posted: 16 Feb 2011, 23:22:56 UTC

Simap appears to be down. I can't report or contact the project in BOINC or by web browser. The site
http://www.downforeveryoneorjustme.com/ also says it is offline when I checked just before posting.
ID: 36885 · Report as offensive
Bernd

Send message
Joined: 24 Aug 09
Posts: 91
United States
Message 36886 - Posted: 17 Feb 2011, 3:15:35 UTC

Simap is back online.
ID: 36886 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37012 - Posted: 25 Feb 2011, 16:05:18 UTC

CPDN main project - important database maintenance on Monday 28 February 2011

The CPDN main project database will offline on Monday in order to facilitate some much-needed maintenance.

During the maintenance period the BOINC forums will be inaccessible, it will not be possible to create accounts or attach new computers to the project and all scheduler requests will fail (this includes uploading trickles, reporting completed tasks and requesting new work). Upload of result files will be unaffected.

The aim is to reduce the time taken for the database backup which is thought to be the current cause both of the BOINC board and scheduler being unavailable for long periods and the slow connections from clients when they are.

The work will take many hours because it involves running the tortuously slow backup script and then archiving old data in the existing (huge) tables.

The database will certainly be offline all of Monday and possibly longer. Updates on the progress will be posted in the News thread on the phpBB forum.
ID: 37012 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37036 - Posted: 28 Feb 2011, 10:56:15 UTC

CPDN main project

The scheduled database maintenance has started and it is anticipated that service will be restored within 36 hours.
ID: 37036 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37041 - Posted: 1 Mar 2011, 10:07:06 UTC

CPDN main project

Unfortunately the backup script (which takes 16 hours to run) failed overnight. It will have to be run again before the planned database maintenance can start, so it will be at least another 36 hours before the database will be running again.
ID: 37041 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37067 - Posted: 3 Mar 2011, 15:28:26 UTC
Last modified: 3 Mar 2011, 15:28:59 UTC

CPDN main project

The database backup has been completed but the maintenance is taking longer than expected. It is now unlikely that the database will be back up before Friday afternoon. If there any further delays the down time could extend into next week.
ID: 37067 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37089 - Posted: 5 Mar 2011, 0:55:46 UTC

CPDN main project

We are now into the final stages of the database maintenance.

Although the BOINC message board is back online the scheduler is still disabled. This means it is still not possible to create accounts or attach new computers to the project and all scheduler requests will continue to fail (this includes uploading trickles, reporting completed tasks and requesting new work).

Upload of most result files is possible, but the final upload file generated by HadAM3P regional tasks (*_13.zip) can't be uploaded at the moment. These files contain the restart dumps required to generate follow-up tasks and are sent to climateapps1.oucs.ox.ac.uk.

By keeping these features disabled the project team can make a direct comparison between the credits calculated before the old database was archived and those calculated using the optimised database.

The project will not be brought fully back to service until the project team are confident that the credit script is working correctly.
ID: 37089 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37114 - Posted: 8 Mar 2011, 12:56:45 UTC

CPDN main project

The scheduler has been restarted and it is now possible to upload trickles, report completed tasks, request new work, create new accounts and attach new computers.

It is still not possible to upload the final file generated by HadAM3P regional tasks (*_13.zip) as climateapps1.oucs.ox.ac.uk is currently out of disk space. Jonathan is working to make more space available.

A significant number of users are currently affected by credit anomalies, mostly with credits below the level calculated before the database maintenance started. We've been here after previous major periods of database maintenance. As before these credit problems will be resolved as a background task by the project team.
ID: 37114 · Report as offensive
Thyme Lawn

Send message
Joined: 2 Sep 05
Posts: 103
United Kingdom
Message 37131 - Posted: 9 Mar 2011, 11:32:46 UTC

CPDN main project

We are fairly sure that the problem which resulted in 5,429 CPDN users losing varying amounts of credit after the database work has been identified. Jonathan is working to fix this.

Jonathan and Milo are still working to make more space available on climateapps1 to allow completion of stalled uploads of HadAM3P regional restart dumps (the *_13.zip files).
ID: 37131 · Report as offensive
Claggy

Send message
Joined: 23 Apr 07
Posts: 1112
United Kingdom
Message 37183 - Posted: 15 Mar 2011, 13:10:28 UTC

Setiathome

Lab-wide Power Outage
Our building's electrical systems are being tested this week. To avoid power surges we will be shutting everything down Tuesday afternoon (after the usual weekly outage) and coming back up Wednesday morning once the tests are finished. All servers/web sites will be down during this time. 14 Mar 2011 | 21:46:40 UTC

Claggy
ID: 37183 · Report as offensive
Bernd

Send message
Joined: 24 Aug 09
Posts: 91
United States
Message 37224 - Posted: 20 Mar 2011, 9:07:11 UTC

SETI is having a bit of a problem with new work distribution. Resends and ghost resends seems to be working. New work and server stats seem to be down for over 24h.
ID: 37224 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 37324 - Posted: 29 Mar 2011, 5:12:43 UTC

Dnetc is 'zero crediting' all workunits -- problem apparently surfaced about 20 hours ago -- it is not clear that the folks over there are aware of the problem as there has been no feedback from Sesef over there regarding this.
ID: 37324 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 37326 - Posted: 29 Mar 2011, 17:22:03 UTC - in response to Message 37324.  

At this point (10am PDT) one can assume that the Dnetc folks are aware of a problem as the site is offline (including the home page). Between the 'no credit' phase and the fully offline phase Dnetc has been in 'outage mode' now for about 34 hours. Hopefully once Dnetc resurfaces it will be running 'normally' and folks will be graced with information regarding the outage.


Dnetc is 'zero crediting' all workunits -- problem apparently surfaced about 20 hours ago -- it is not clear that the folks over there are aware of the problem as there has been no feedback from Sesef over there regarding this.

ID: 37326 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 37408 - Posted: 6 Apr 2011, 13:37:59 UTC - in response to Message 37326.  

The folks over at Dnetc were able to correct problems (both hardware and software) after about 1 day and a half last week, but seem to have encountered either a repeat of the hardware problem or something equally showstopping. As a result, the project has been totally offline for the past 30 hours. Like a number of the smaller projects, when they go offline, that includes even the home page. Hopefully they will get things resolved and in a way that doesn't leave the project in fragile mode.

At this point (10am PDT) one can assume that the Dnetc folks are aware of a problem as the site is offline (including the home page). Between the 'no credit' phase and the fully offline phase Dnetc has been in 'outage mode' now for about 34 hours. Hopefully once Dnetc resurfaces it will be running 'normally' and folks will be graced with information regarding the outage.


Dnetc is 'zero crediting' all workunits -- problem apparently surfaced about 20 hours ago -- it is not clear that the folks over there are aware of the problem as there has been no feedback from Sesef over there regarding this.


ID: 37408 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 37410 - Posted: 6 Apr 2011, 15:30:30 UTC

MilkyWay has joined the ATI projects with problems -- they have had no new work for ATI GPU's for the past day.
ID: 37410 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 37411 - Posted: 6 Apr 2011, 16:53:20 UTC - in response to Message 37410.  

Good then that I run them on my CPU these days only. Still got one of their multi-threaded tasks to work through. ;-)
ID: 37411 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 . . . 10 · Next

Message boards : Projects : News on Project Outages

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.