CPDN Main Project Upload Problem

uploader.oerc.ox.ac.uk is back up but Milo is not sure how stable it is going to be (a disk in the RAID array was showing a SMART failure). He is contacting the manufacturer for support.

CPDN main project upload outage

The disk of upload server uploader.oerc which caused earlier problems has failed completely. Milo ordered a replacement disk and must now install it. As during the previous outage, the server may display as running on the Server status page even though it is down.

The outage will affect FAMOUS upload files 3, 6, 9, 12, 15 and 18. If other files become stuck in Transfers please see Thyme Lawn's advice two posts above this.

You can choose whether to suspend Boinc network activity or take no action.

Whats happening with Goldbach's Conjecture? It has been down for about a week or more.

---

CPDN uploads

Uploader.oerc is running and accepting uploads.

CPDN main project uploads

As you can see from the Server status page upload server uploader.oerc is down. Its disk is nearly full so Milo is transferring data from it. Affected uploads will be FAMOUS files 3, 6, 9, 12, 15 and 18. Files with other numbers upload to different servers and will not be affected.

You can either suspend Boinc network activity or take no action.

DNetc went offline yesterday at about 1PM PDT for a planned software update. The estimate was for a five hour outage. For several hours after that the home page and message boards were live. At about 5PM, they went offline and the site (at least til now - 8AM Friday PDT) is totally offline.

MW went offline sometime after 7AM this morning -- totally offline. For MW, this sort of thing happens often enough when the good folks at RPI are working at reconfiguring power to the building -- something they do unusually frequently.

DNetc went offline yesterday at about 1PM PDT for a planned software update. The estimate was for a five hour outage. For several hours after that the home page and message boards were live. At about 5PM, they went offline and the site (at least til now - 8AM Friday PDT) is totally offline.

From Message 89655 at the BOINCStats News forum:

DNETC RAID Failure
One of the raid hdd's have died in half of update process - we are trying to restore project from backup. Project off - for one day (28.08)

Gruß,
Gundolf

Ah --- thanks for that. I hope they were at least in RAID 5 mode (ideally RAID 5 plus a hot spare -- as disks are not that expensive). Then again, I know some folks love the speed of a multi-drive RAID 0 setup....

DNetc went offline yesterday at about 1PM PDT for a planned software update. The estimate was for a five hour outage. For several hours after that the home page and message boards were live. At about 5PM, they went offline and the site (at least til now - 8AM Friday PDT) is totally offline.

From Message 89655 at the BOINCStats News forum:

DNETC RAID Failure
One of the raid hdd's have died in half of update process - we are trying to restore project from backup. Project off - for one day (28.08)

Gruß,
Gundolf

CPDN main project uploads

Uploader.oerc is up and running.

Looks like Dnetc is having further trouble with the rebuild. Late last night they had their home page address up (with no content) so it lloked like some progress. Now, eight hours later, they are back to their start point. Perhaps another drive failure???

OK -- Dnetc repeated 24 hours later the same cycle, home up with no content, then 8 hours later, back to the same 'dead' start point. Perhaps their back ups are corrupt as well.

Looks like Dnetc is having further trouble with the rebuild. Late last night they had their home page address up (with no content) so it lloked like some progress. Now, eight hours later, they are back to their start point.

Perhaps the folks at Dnetc are going to have to start all over from scratch -- looks like their various recovery efforts are not meeting with success at all.

Still no follow up from Dnetc -- they have a placekeeper up at their home page -- but nothing further over the past 24 hours. I do hope they can get their project back up, since it is one of only two projects with broad based (single precision and double precision CUDA and ATI) GPU support.


Index of /
[ICO] Name Last modified Size Description
[DIR] webalizer/ 30-Aug-2010 15:16 -
Apache/2.2.16 (Debian) Server at dnetc.net Port 80

You guys can always go to http://www.distributed.net and download the Stream client in the meantime. The keyrate for RC5 is down 75%.

Dnetc is working through a full set of issues -- hardware first (Hard drive and memory), then software (working with a restore from backup). They hope to be back online later in the week.

CPDN main project

Milo is moving data from server disks so they don't fill up. There will be some upload server outages. Here is the Server status page.

CPDN main project

Milo has temporarily shut the project down to perform a database archive.

This means that all scheduler requests will fail (including trickles, work requests, reporting completed tasks and attaching to the project) and the BOINC forums are inaccessible.

uploader.oerc is also unavailable while more space is being made available by moving files elsewhere. This affects FAMOUS upload files 3, 6, 9, 12, 15 and 18.

Both MW and Dnetc are back up and running -- good news there for GPU folks.

In addition to the ongoing storage issues for Climate, another of the larger projects (Einstein) has run into a problem at the server side - no uploads or downloads until they figure out the problem (or perhaps simply stop/start the server).

Seti and Seti Beta are down:

The air conditioning in our server closet failed this morning, causing several of our important systems to overheat. The servers have since been shut down, but until we survey the damage and address these cooling issues all parts of the project will remain offline.

Claggy

CPDN main project

Milo has had to disable upload climateapps1.oucs which is an upload server that takes regional model files. Its disk is full and data will need to be copied from it tomorrow.

If you have HadAM3P regional model files that cannot upload you could temporarily turn off Boinc network activity (in the Activity menu). Or you could take no action.

Here is the Server status page.