News on project outages

Message boards : Projects : News on project outages
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 10 · Next

AuthorMessage
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29003 - Posted: 26 Nov 2009, 0:22:52 UTC

Continuing in new thread.

Forward warning of little to no new work on Seti@Home. As per this post by Matt Lebofsky:

... until we ran out of work to send out. Turns out the last 10 files I brought up from Arecibo are all broken. <sad trombone>Fwa wa wa waaaaa</sad trombone>. This is particularly frustrating as I was busting my hump trying to get enough work on line before the long holiday weekend, and now we have zero. So it'll be to me and Jeff to check in over the next few days and kick the pipeline along. We'll be out of real work to send out until this evening at the earliest, and quite probably hit long periods of no work throughout the weekend. Fine.

From Thursday 26-11-09 till Sunday 29-11-09 it's the Thanksgiving long weekend in the USA. Give the guys at Seti some chance to celebrate it as well.
ID: 29003 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29015 - Posted: 26 Nov 2009, 17:22:55 UTC - in response to Message 29003.  

Seti continued....

Matt Lebofsky wrote:
Oh well, we tried. We thought we would just have to put some extra minutes monitoring the data pipeline over the weekend (after wasting a lot of time bringing up many broken files), which wouldn't have been too bad, but...

Then bambi crashed last night - it's our secondary science database server but also manage a lot of the data pipeline stuff. I happened to be free so I drove up to the lab around 10:30pm and rebooted it. After that, the pipeline zipped right along.

That is... until 11pm when the router up and died. Or something along the entire Hurricane Electric network path died. We have no idea. Jeff and I fought with it (both remotely) this morning, but we're throwing up our hands at this point and going on holiday.

Might as well have everything fail at once, and at the start of a long holiday weekend. Why not?

- Matt

So Seti is effectively down for the rest of the week.
ID: 29015 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29049 - Posted: 27 Nov 2009, 20:01:09 UTC

Seti continued...

Their front page news wrote:
November 27, 2009
The connectivity problem noted yesterday turned out to be one of our switches and not the router. We swapped in a replacement switch today and connectivity was restored.

Downloads are now trying to get through.
ID: 29049 · Report as offensive
mo.v
Avatar

Send message
Joined: 13 Aug 06
Posts: 778
United Kingdom
Message 29119 - Posted: 1 Dec 2009, 22:40:19 UTC

CPDN main project

The disk of upload server cpdn-uploader1.comlab almost filled up today. Milo has shut it down to remove data and make space. This server takes the second of the three final files produced by HadAM3P models. Files #1 and #3 should upload without problems as they go to different servers but #2 will remain in the Boinc Transfers tab. Do not try to to force it to upload as it cannot at the moment.

You may wish to suspend Boinc network activity until cpdn-uploader1.comlab is up again, probably in a day or two. Or you could do nothing. The server status page is here.
ID: 29119 · Report as offensive
mo.v
Avatar

Send message
Joined: 13 Aug 06
Posts: 778
United Kingdom
Message 29175 - Posted: 3 Dec 2009, 16:04:30 UTC

CPDN main project

Milo has reactivated upload server cpdn-uploader1.comlab so any HadAM3P #2 files waiting in the Transfers tab can now upload.

Reminder: subscribe to the News thread on one of the CPDN forums to keep up-to-date. That's where we try to announce everything that members need to know.
ID: 29175 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 29186 - Posted: 3 Dec 2009, 19:46:46 UTC

After 10 solid days, Collatz seems to have gone walkabout -- I hadn't tracked their homepage over the past couple of days, so I don't know if this outage is a planned one.
ID: 29186 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 29230 - Posted: 4 Dec 2009, 22:40:59 UTC - in response to Message 29186.  

And still offline 30 hours later.....

After 10 solid days, Collatz seems to have gone walkabout -- I hadn't tracked their homepage over the past couple of days, so I don't know if this outage is a planned one.

ID: 29230 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 29240 - Posted: 5 Dec 2009, 7:36:46 UTC - in response to Message 29230.  

Make that 40 hours for Collatz -- I'm hoping that whatever took Slicker away from the project (I'm assuming that the extended outage is a combination of one or more problems with his server AND him being called away to other matters) is both a short term event and one not strewn with bad news (personal or project wise).
ID: 29240 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 29261 - Posted: 5 Dec 2009, 18:12:48 UTC - in response to Message 29240.  

Make that something like 56 hours now -- Collatz went offline about 3AM on Thursday morning (PST). No information as to the nature of the problems there has been coming from Slicker -- I am guessing that he hasn't been around to diagnose and repair whatever ails the server (or internet connectivity). I hope he is ok.


Make that 40 hours for Collatz -- I'm hoping that whatever took Slicker away from the project (I'm assuming that the extended outage is a combination of one or more problems with his server AND him being called away to other matters) is both a short term event and one not strewn with bad news (personal or project wise).

ID: 29261 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29277 - Posted: 6 Dec 2009, 0:30:30 UTC
Last modified: 6 Dec 2009, 0:30:46 UTC

Seti@Home is down for maintenance. The forums and scheduler are off line. And it's not even Tuesday.
ID: 29277 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 29284 - Posted: 6 Dec 2009, 15:27:54 UTC - in response to Message 29277.  

Perhaps it will be down for 'database reconfiguration' until Tuesday when it will be down for maintenance.....

I have the feeling that 'database reconfiguration' is a term of art, like 'unsupercalifragilisticexpealidocious'. It is a way of conveying some unforseen technical problem that might not at the moment be understood.


Seti@Home is down for maintenance. The forums and scheduler are off line. And it's not even Tuesday.

ID: 29284 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29285 - Posted: 6 Dec 2009, 19:05:24 UTC - in response to Message 29284.  

Seti's forums are back, but all the rest is still down.
ID: 29285 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29286 - Posted: 6 Dec 2009, 22:17:43 UTC - in response to Message 29285.  

Seti@home:
Temporary Outage
Our database work has progressed to the point where we are bringing up SETI@Home. Astropulse remains offline.

ID: 29286 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29302 - Posted: 8 Dec 2009, 1:51:46 UTC - in response to Message 29286.  

Seti@home:
Temporary Outage
Both SETI@Home and Astropulse work production have been halted in order to give database resources to the repairing of a large Astropulse table.
ID: 29302 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29331 - Posted: 9 Dec 2009, 21:59:50 UTC - in response to Message 29302.  
Last modified: 9 Dec 2009, 21:59:59 UTC

Seti@Home,

Seti_Enhanced work is available.
Astropulse will stay off line.
ID: 29331 · Report as offensive
BarryAZ

Send message
Joined: 4 Sep 09
Posts: 381
United States
Message 29343 - Posted: 10 Dec 2009, 15:47:34 UTC

Einstein project appears offline -- can't access home page or servers.
ID: 29343 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29345 - Posted: 10 Dec 2009, 17:53:34 UTC - in response to Message 29343.  

Einstein News wrote:
December 10, 2009
Parts of the project were taken offline for some cleanup necessary to fix the recent scheduler http problems. We hoped that this could be done within minutes and thus remain unnoticed for most users, but it looks like it will take another hour or two until the scheduler is running again.
ID: 29345 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14771
Netherlands
Message 29352 - Posted: 10 Dec 2009, 22:41:00 UTC - in response to Message 29345.  

And their database is down again. ;-)

Einstein News wrote:
December 10, 2009
Parts of the project were taken offline for some cleanup necessary to fix the recent scheduler http problems. We hoped that this could be done within minutes and thus remain unnoticed for most users, but it looks like it will take another hour or two until the scheduler is running again.

ID: 29352 · Report as offensive
ZPM
Avatar

Send message
Joined: 14 Mar 09
Posts: 215
United States
Message 29405 - Posted: 13 Dec 2009, 7:25:58 UTC - in response to Message 29352.  

GPUGRID is offline from every aspect i've looked at.... switching to back up project collatz and seti for nvidia gtx 260 FOC.
ID: 29405 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4641
United Kingdom
Message 29406 - Posted: 13 Dec 2009, 10:56:19 UTC - in response to Message 29405.  

GPUGRID is offline from every aspect i've looked at.... switching to back up project collatz and seti for nvidia gtx 260 FOC.

They had a powercut. They're back now, but warning of recovery problems - database server seems to be up and down every few seconds.
ID: 29406 · Report as offensive
1 · 2 · 3 · 4 . . . 10 · Next

Message boards : Projects : News on project outages

Copyright © 2021 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.