Loss of statistical data when machine crashes

Message boards : Questions and problems : Loss of statistical data when machine crashes
Message board moderation

To post messages, you must log in.

AuthorMessage
Cliff Harding
Avatar

Send message
Joined: 26 Jun 14
Posts: 26
United States
Message 104808 - Posted: 16 Jul 2021, 14:42:22 UTC

I've noticed that when/if my machine crashes, mostly heat related, I lose my accumulated stats on the stats tab of the manager. It does not happen often but it does happen, and the stats start all over again from the last point of reference prior to the crash. The daily stats file for the project does not seem to be affected, just the STATISTICS tab on the manager. The stats argument on the cc_config_xml file is set to 365 days and has been for several years. So somewhere the accumulated stats are being lost. Please note that this happens regardless what the length of the stats argument.

Currently I'm running 7.17.0 (x64) on Win 10 (build: 19043.1110), which is a prerelease, but this goes back much further; prior to 7.16.11.
ID: 104808 · Report as offensive
Harri Liljeroos

Send message
Joined: 25 Jul 18
Posts: 23
Finland
Message 104809 - Posted: 16 Jul 2021, 18:01:58 UTC

I have this happened to me a few times. I am running currently three projects and this doesn't happen to all projects. Usually only one of them loses the stats and others remain unharmed. I am currently running 7.16.5 on win 10 and win 7 machines.
ID: 104809 · Report as offensive
Cliff Harding
Avatar

Send message
Joined: 26 Jun 14
Posts: 26
United States
Message 104810 - Posted: 16 Jul 2021, 18:11:24 UTC

You're lucky, when I was running multiple projects I lost everything on each project.
ID: 104810 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 14767
Netherlands
Message 104811 - Posted: 16 Jul 2021, 18:54:58 UTC - in response to Message 104808.  

Currently I'm running 7.17.0 (x64)
Which doesn't say much. Whenever you downloaded the source code during the past 7.16 development cycle (starting April 2020) and compiled it, it would compile as 7.17.0 as that's what it says in the versions.h header file. And if you don't change it, it'll be that version.

So if you want to report something like that, say when you downloaded its code.
Also, as with everything computers: backup, backup, backup. So, backup your data directory every week, in such case you lose a week at max at a next crash. Of course, you can also run a script that backs up the statistics_*.xml files every day.
ID: 104811 · Report as offensive
Cliff Harding
Avatar

Send message
Joined: 26 Jun 14
Posts: 26
United States
Message 104812 - Posted: 16 Jul 2021, 20:52:40 UTC - in response to Message 104811.  

Currently I'm running 7.17.0 (x64)
Which doesn't say much. Whenever you downloaded the source code during the past 7.16 development cycle (starting April 2020) and compiled it, it would compile as 7.17.0 as that's what it says in the versions.h header file. And if you don't change it, it'll be that version.

So if you want to report something like that, say when you downloaded its code.
Also, as with everything computers: backup, backup, backup. So, backup your data directory every week, in such case you lose a week at max at a next crash. Of course, you can also run a script that backs up the statistics_*.xml files every day.


The BOINCMGR & BOINCTRAY .exes were compiled by Richard Haselgrove on 29 Oct, 2020 on one of his machines and sent to me on 26 Jan, 2021. This was in regards with the problems with the 7.16.16 installer. I immediately installed them replacing the same .exe files with them in the BOINC program files on my system drive (C:).

This is what stats file, which physically resides on my data drive (D:\BOINC) looks like for the project that I'm currently running. As you can see the individual stats file for this project was not affected by the crash, and it never has as far as I can determine. It would be the same if I were to have multiple projects on the machine. The problem lies in how it is displayed on the statistics tab on the manager. What should have been a trace line of daily work for several months has disappeared, as this file gets updated several times each day as long as the project is active. If I had multiple projects on my machine there would be stats for each project. As I stated this goes back further than 7.16.x.

If a project was suspended for a period of time it would be reflected in the downward slope of its trace line.

<project_statistics>
<master_url>http://milkyway.cs.rpi.edu/milkyway/</master_url>
<daily_statistics>
<day>1626393600.000000</day>
<user_total_credit>80385628.041079</user_total_credit>
<user_expavg_credit>200882.273097</user_expavg_credit>
<host_total_credit>78317523.785721</host_total_credit>
<host_expavg_credit>200882.262785</host_expavg_credit>
</daily_statistics>
</project_statistics>
ID: 104812 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 4635
United Kingdom
Message 104813 - Posted: 16 Jul 2021, 21:38:30 UTC

That particular sequence of events started with the unexpected and unexplained appearance of a v7.16.16 installer in the download directory at 2020-10-27 19:45. That turned out to be a VS2019 experiment, built from whatever happened to be in the release branch at the time. That's not a flippant remark: the release branch is supposed to be a tightly controlled, reference version that satisfies the needs of the LGPL 'source code' condition. The v7.16 branch - hasn't, to put it mildly,

I won't look further into the details at this time on a Friday night (UK time), but I'm pretty sure no changes have been made to the statistical record keeping in recent years.

The statistics for each project are kept in a separate file. I think (subject to checking) that each file is re-written from scratch - using the data held in RAM - each time that project's data changes, usually when a project task completes. That's the only time the data in the file is vulnerable to errors.

The fundamental issue would be - your computer shouldn't crash. External factors, like a local thunderstorm affecting the electricity supply, can't easily be avoided, but apart from that, modern computers should run for months without errors.

If a particular project should regularly crash the machine - then that's a bad project. Avoid running it on that machine. Even saying that, a bad project should only mess up its own statistical record, because only that single file should be being re-written at the moment the crash occurs.

We can explore this further - after I've had a night's sleep! - but we would need further information about which projects are being run, and the nature of the crashes that trigger the problem.
ID: 104813 · Report as offensive

Message boards : Questions and problems : Loss of statistical data when machine crashes

Copyright © 2021 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.