Undocumented change to stats export function?

Message boards : Web interfaces : Undocumented change to stats export function?
Message board moderation

To post messages, you must log in.

AuthorMessage
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4598
United Kingdom
Message 19310 - Posted: 7 Aug 2008, 23:29:14 UTC

Several projects have been updating their server code recently - major updates at LHC and Einstein, more incremental ones at CPDN and SETI.

Each of those four has suffered from delays in transferring statistics to external stats sites. As I type, the 'last update' is showing as:

LHC - Mon 14 July
Einstein - Sat 02 Aug

at both BOINCstats and BOINC Synergy.

Einstein exported around 14:00 UTC today - Bernd posted, and I checked that the folder which was previously empty is now populated.

There's an interesting post by Toby (KSWN) at LHC:

Looks like the names of the XML files changed (from team_id.gz to team.gz and same for host and user). I just updated my database to reflect this and it seems to be working again.

They couldn't have .................

............, could they?
ID: 19310 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1179
Argentina
Message 19333 - Posted: 8 Aug 2008, 13:56:08 UTC - in response to Message 19310.  

There's an interesting post by Toby (KSWN) at LHC:

Looks like the names of the XML files changed (from team_id.gz to team.gz and same for host and user). I just updated my database to reflect this and it seems to be working again.

They couldn't have .................

............, could they?

Well, you aren't supposed to hard-code the XML file name anyway. You're supposed to parse db_dump.xml to know what the files are. <sarcasm>Didn't you see that in the documentation??</sarcasm>

AFAIK, Einstein has had the stats filenames different from the other projects like forever.
ID: 19333 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4598
United Kingdom
Message 19347 - Posted: 8 Aug 2008, 15:46:33 UTC - in response to Message 19333.  

Well, you aren't supposed to hard-code the XML file name anyway. You're supposed to parse db_dump.xml to know what the files are. <sarcasm>Didn't you see that in the documentation??</sarcasm>

AFAIK, Einstein has had the stats filenames different from the other projects like forever.

It's interesting to note:

1) When Einstein changed the file names back to their previous ones, BOINC Synergy picked up the update OK, thus, I think, confirming my theory that the name-change caused the problem in the first place. (Bruce and Willy are chasing each other all over the file name-space at BOINCstats, so that may take a little longer to settle down).

2) The cross-project stats shown on the bottom of everyone's account pages at every BOINC project are also broken - the Einstein one was stuck at the Sat 2nd August figure until today. Are you suggesting that the officially-sanctioned, source-coded stats repository wasn't written according to the documentation, either?
ID: 19347 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1179
Argentina
Message 19352 - Posted: 8 Aug 2008, 20:13:57 UTC - in response to Message 19347.  

Are you suggesting that the officially-sanctioned, source-coded stats repository wasn't written according to the documentation, either?

With my sarcasm, I'm suggesting there is no documentation at all. Developers of stats sites have nowhere to read about what to rely on and what not.

If there was documentation saying "don't rely on filenames, parse db_dump.xml instead", then if one day filenames changed, we could blame stats sites for not doing things like the docs said they should. But there isn't any documentation, not even about the file format of user/host/team.gz.

This is an undocumented change on an undocumented protocol.

ID: 19352 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1179
Argentina
Message 19353 - Posted: 8 Aug 2008, 20:21:53 UTC - in response to Message 19347.  

Are you suggesting that the officially-sanctioned, source-coded stats repository wasn't written according to the documentation, either?

I misunderstood your question.

Yes, the officially-sanctioned, source-coded stats scripts in the official BOINC SVN repository hard-code full URLs to the XML files on each project.

http://boinc.berkeley.edu/trac/browser/trunk/boinc_stats/boinc_combined_stats/scripts/fetch_files.sh

But my point still stands that there isn't any "documentation to write according to".
ID: 19353 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4598
United Kingdom
Message 19354 - Posted: 8 Aug 2008, 20:39:38 UTC

Well, there's a reasonable start to a documentation framework at http://boinc.berkeley.edu/trac/wiki/XmlStats, but as you say no reference to the db_dump.xml construct.

The 'host' xml structure is also outdated - fields <rpc_time> and <credit_per_cpu_sec> are undocumented. I haven't needed to download a recent copy of the other files, so I haven't checked them.

Judging by the angst at Einstein (see No xml stats update since upgrade), and your opening remarks here, the actual undocumented change took place a long time ago, and the decision by the various stats sites to use fixed file names, rather than using the db_dump.xml to extract names which one might reasonably expect to be slowly-changing, is probably just as ancient.

Anyway, Willy's drive to bring BOINCstats fully up-to-date has found me an extra 0.11 credits I didn't know I had (awarded at LHC since 14 July), so it can't all be bad.

But the "Projects in which you are participating" cross-project stat for Einstein still hasn't updated.
ID: 19354 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4598
United Kingdom
Message 19355 - Posted: 8 Aug 2008, 20:46:29 UTC - in response to Message 19353.  

Yes, the officially-sanctioned, source-coded stats scripts in the official BOINC SVN repository hard-code full URLs to the XML files on each project.

http://boinc.berkeley.edu/trac/browser/trunk/boinc_stats/boinc_combined_stats/scripts/fetch_files.sh

Which hasn't been updated since 05/03/07 - I read that as 3rd. May 2007, or 15 months ago. Time for a review/recheck, if projects like LHC have updated to export code which says something different? And have there really been no new projects in all that time?
ID: 19355 · Report as offensive
Nicolas

Send message
Joined: 19 Jan 07
Posts: 1179
Argentina
Message 19360 - Posted: 8 Aug 2008, 22:18:22 UTC - in response to Message 19333.  

I'm reading the db_dump.C code.

Did anyone see my sanity around? I seem to have lost it.

ID: 19360 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14730
Netherlands
Message 19361 - Posted: 8 Aug 2008, 22:32:28 UTC - in response to Message 19360.  

I think I saw it in the cupboard over there underneath the 1955 version of the Encyclopedia Brittanica. Both quite dusted over.

ID: 19361 · Report as offensive

Message boards : Web interfaces : Undocumented change to stats export function?

Copyright © 2021 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.