Message boards : Projects : News on Project Outages
Message board moderation
Previous · 1 . . . 46 · 47 · 48 · 49 · 50 · 51 · 52 . . . 67 · Next
Author | Message |
---|---|
Send message Joined: 30 Mar 20 Posts: 408 |
No further news from the WCG team. Still down, very very deep down...... Since it's almost 23:00 hours, in Toronto now, WCG is probably going to stay down, at least over the (Canadian) night. |
Send message Joined: 25 Aug 08 Posts: 40 |
No further news from the WCG team. Still down, very very deep down...... The lack of news (and progress) is somewhat concerning. Also, why would the forums be down at the same time? I thought they were hosted separately. |
Send message Joined: 10 May 07 Posts: 1436 |
The lack of news (and progress) is somewhat concerning. It was when IBM ran the show. Don't know how much physical hardware Krembil consolidated project on, but my guess is it's a lot less than IBM. Being 30+ hours since last Facebook update/post from Krembil, who have not been good at communicating things since they took over 18 months ago... We don't have a clue what's happening. |
Send message Joined: 3 Mar 23 Posts: 10 |
To: Grumpy Swede Thank you for posting updates on WCG here. I don't use Facebook or Twitter so, with the forums down as well, this thread has become my source of information. Cheers Phillip |
Send message Joined: 30 Mar 20 Posts: 408 |
To: Grumpy SwedeNo problems Phillip. I do not "use" Facebook or Twitter either (I have no accounts), but one can always read the postings on most public Facebook and Twitter accounts. Sadly though, it has not been any more updates on the WCG FB account. They really need to improve their communication skills. This silence towards us volunteers, are simply not acceptable. Just a few words would make a big difference. Like "We're having big problems with the RAID array, and it will take about xx hours/days to rebuild the array", or "The entire RAID array is toast, and we have to rebuild it from backups, and it will take about xx hours/days" (Let's hope their backups has been handled in a professional way.) To have this kind of SPOF (Single Point Of Failure), in such a big system as WCG, is just ridiculously unprofessional. When/if, they recover from this, the first thing they need to do, is hire a real IT-professional, to look through their entire setup, and identify other weaknesses, and other SPOF's. |
Send message Joined: 30 Mar 20 Posts: 408 |
Just in, New Update: "Update #2: The borrowed RAID card worked and the drive layout was recognized, so we have all data safe (there is also a tape backup, but accessing that would be slower). Data center managed a full boot and we expect we will resume operation later today." Edit, added: the last part "we expect we will resume operation later today", is something I would take with more than one grain of salt |
Send message Joined: 28 Jun 10 Posts: 2672 |
To have this kind of SPOF (Single Point Of Failure), in such a big system as WCG, is just ridiculously unprofessional. When/if, they recover from this, the first thing they need to do,I suspect the real issue is the amount of cash going in is a fraction of what there was when IBM had it. If they had more cash, I am sure their infrastructure would have more redundancy built in. |
Send message Joined: 30 Mar 20 Posts: 408 |
New Update, about 4-hours ago: "Update #3: All disks have finished rebuilding and the MySQL and DB2 filesystems are mounted. We are currently checking the /science filesystem integrity." WCG is still down though...... |
Send message Joined: 30 Mar 20 Posts: 408 |
New update, just in: "Update #4: We have confirmed all the data is intact and have replaced the RAID controller, but we are still having some issues with getting the new hardware production ready. Unfortunately, data center staff will not be able to help us over the weekend. Additionally, the deadline of all existing WUs that are partially done will be extended and accepted once the hardware change is done." So, as I expected, WCG will likely continue to be down, until at least some time next week. (I wouldn't be surprise though, if WCG was down for another week, or two) |
Send message Joined: 30 Mar 20 Posts: 408 |
A slight difference now on the WCG home page. You no longer get the normal homepage, but instead directly the Errror "System error World Community Grid is currently experiencing an unexpected error. Please check Facebook or Twitter for more information." Also the forum ink (or other links needing access to a database) , does no longer show "Error 500: javax.servlet.ServletException: net.myvietnam.mvncore.exception.DatabaseException: Error executing SQL in MVNForumPermissionWebHelper.getPermissionsForGroupGuest." The forum link now also shows: "System error World Community Grid is currently experiencing an unexpected error. Please check Facebook or Twitter for more information." So, somebody is possibly up early in Toronto, trying to perhaps restart the system, and restore connections to the various databases, and filesystems. Edit, added: No life from BOINC yet though. BOINC is still replying to a request with "Server error: feeder not running" |
Send message Joined: 3 Mar 23 Posts: 14 |
Thx for info ; ) |
Send message Joined: 30 Mar 20 Posts: 408 |
And it's finally Monday. Let's see if anything at all happens with WCG today. It's only 05:33 in Toronto yet, so maybe something happens in a couple of hours. |
Send message Joined: 3 Mar 23 Posts: 10 |
And it's finally Monday. Let's see if anything at all happens with WCG today. More in hope than certainty! Do you have pigs that fly in Sweden? I just saw several Porcine Flying Objects overhead in South West France and distinctly heard them squealing "(B)oink, (B)oink" Fingers crossed! Phillip |
Send message Joined: 28 Jun 10 Posts: 2672 |
More in hope than certainty! Do you have pigs that fly in Sweden? I just saw several Porcine Flying Objects overhead in South West France and distinctly heard them squealing "(B)oink, (B)oink"I don't know the velocity of flying pigs, they may take a while to reach Toronto. |
Send message Joined: 30 Mar 20 Posts: 408 |
Well, we do have some flying pigs here in Sweden, most of them are politicians :-) Yeah, fingers crossed for WCG returning before the next ice age. |
Send message Joined: 25 May 09 Posts: 1298 |
Only a matter of hours until the next ice age in the north of the UK...... |
Send message Joined: 8 Feb 17 Posts: 7 |
I'm very upset over the current state of affairs. I've been crunching files since the start of WCG. I'm now doing Rosetta files hoping WCG will come back soon. Glad to have found this forum since FB and the WCG forum are down. |
Send message Joined: 30 Mar 20 Posts: 408 |
New Update from WCG on Facebook, a couple of minutes ago: "Hello everyone, hope you had a great weekend. We are still working with data centre to resolve the hardware failure so we can restart the storage, BOINC and website ASAP. We will post updates as we receive them. Thank you for your patience." |
Send message Joined: 10 May 07 Posts: 1436 |
... Another day wasted because Krembil's incompetence and running around chasing it's tail :( |
Send message Joined: 30 Mar 20 Posts: 408 |
It's soon 14:00 hours in Toronto, and I seriously doubt that we'll see any working WCG today either. Sigh...... |
Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.2 or any later version published by the Free Software Foundation.