BOINC doesn't write in-progress downloads to disk?

Message boards : BOINC client : BOINC doesn't write in-progress downloads to disk?
Message board moderation

To post messages, you must log in.

AuthorMessage
funkydude

Send message
Joined: 20 Dec 08
Posts: 12
United Kingdom
Message 52692 - Posted: 21 Feb 2014, 14:26:08 UTC

I'm currently part of the BURP project, which is experiencing severe bandwidth issues due to upgrading their infrastructure. That means it's taking around a day just to download a 300MB file to be rendered.

I was about halfway through the download when unfortunately my "high quality" AMD drivers kicked in and bluescreened my PC. To my surprise when I relaunched BOINC, the download started again from 0%.

Are downloads not periodically saved to disk in BOINC?
ID: 52692 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5081
United Kingdom
Message 52693 - Posted: 21 Feb 2014, 14:37:37 UTC - in response to Message 52692.  

I think they are, and can be resumed from part-way.

There is a possible problem with compressed (zipped) files, and I suspect (without knowledge) that the BURP files are probably zipped. If the files are zipped in advance, and stored on the project server in their compressed state, downloads can be interrupted and resumed normally. But - initially at least - files which were compressed 'on the fly' as they were downloaded couldn't be resumed: they had to be restarted from the beginning. So, if they use that technique, it slightly depends if both you and the project are using recent enough BOINC software to handle resumptions of compressed-on-the-fly files.

One further possibility is BOINC's display of the size and progress of resumed downloads. I've never summoned up the energy to investigate in detail, but I have a suspicion the BOINC treats a resumed file as a complete, smaller, file for progress display purposes. In other words, instead of showing you a 300MB file resuming at 50%, it treats it as a 150MB file starting from 0%. You might be able to confirm that observation? If true, I agree it's confusing.
ID: 52693 · Report as offensive
funkydude

Send message
Joined: 20 Dec 08
Posts: 12
United Kingdom
Message 52717 - Posted: 22 Feb 2014, 17:53:12 UTC - in response to Message 52693.  
Last modified: 22 Feb 2014, 17:53:50 UTC

This is the response the BURP admin gave:
BURP uses pre-computed zips and does support both HTTP 206 Partial Content replies as well as most commonly used Cache Control headers for caching and proxying.


I just bluescreened again, this time the first thing I did was look at BOINC. It had started at 11MB, even though I had completed nearly 200MB of the download.

That means BOINC only wrote 11MB to disk. This seems like a bug in BOINC. I am using the latest version available, always.

Unfortunately I've now been forced to abandoned 4 tasks due to this.
ID: 52717 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 52718 - Posted: 22 Feb 2014, 18:23:31 UTC - in response to Message 52717.  
Last modified: 22 Feb 2014, 18:28:42 UTC

What does the blue screen say? You can check that with Blue Screen View.

As it would be better to solve the cause of the blue screen interruptions first.
I'll ask the developers about how BOINC saves big files.

Apropos, with 'the latest BOINC' you mean which one? 7.2.33, 7.2.39, 7.3.2?
ID: 52718 · Report as offensive
funkydude

Send message
Joined: 20 Dec 08
Posts: 12
United Kingdom
Message 52720 - Posted: 22 Feb 2014, 19:26:48 UTC - in response to Message 52718.  

What does the blue screen say? You can check that with Blue Screen View.

As it would be better to solve the cause of the blue screen interruptions first.


Not really. This issue isn't a direct result of the Blue Screens, at least I don't think so. That's just AMDs typical approach towards testing drivers for cards that aren't their latest 2 generations. Reverting to earlier drivers would be worse for me than the occasional BSOD unfortunately. I'll fix the Blue Screens when I ditch AMD later this year.

I'll ask the developers about how BOINC saves big files.


Does it treat larger files differently? :o


Apropos, with 'the latest BOINC' you mean which one? 7.2.33, 7.2.39, 7.3.2?


Sorry, I meant latest stable, which is 7.2.39 currently.
ID: 52720 · Report as offensive
funkydude

Send message
Joined: 20 Dec 08
Posts: 12
United Kingdom
Message 53442 - Posted: 31 Mar 2014, 20:28:54 UTC - in response to Message 52720.  

Did anything happen of this? I fixed the BSOD's on my side by moving to NVIDIA.
ID: 53442 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15480
Netherlands
Message 53443 - Posted: 31 Mar 2014, 21:18:27 UTC - in response to Message 53442.  

No, nothing happened of it. As you already say, you fixed it by fixing the cause of the BSODs, which is what I was after in my previous post in this thread. So there's nothing for BOINC to fix.
ID: 53443 · Report as offensive
funkydude

Send message
Joined: 20 Dec 08
Posts: 12
United Kingdom
Message 53449 - Posted: 1 Apr 2014, 1:54:42 UTC - in response to Message 53443.  

No, nothing happened of it. As you already say, you fixed it by fixing the cause of the BSODs, which is what I was after in my previous post in this thread. So there's nothing for BOINC to fix.


Did you read the original post or are you screwing with me for April fools? The issue was never the BSOD...
ID: 53449 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 321
Germany
Message 53454 - Posted: 1 Apr 2014, 7:55:07 UTC

Hi funkydude,

did you encounter the download problem since you solved the problem with the BSOD?

Resuming partial downloads is only necessary if something unexpected (like a BSOD) happens during download. It 's also a little hard to reproduce and track down if there is a bug in the code. If you still have the problem and we can somehow reproduce it on a dev system we can try to fix it.
ID: 53454 · Report as offensive
funkydude

Send message
Joined: 20 Dec 08
Posts: 12
United Kingdom
Message 53459 - Posted: 1 Apr 2014, 13:44:49 UTC - in response to Message 53454.  
Last modified: 1 Apr 2014, 13:45:16 UTC

Hi funkydude,

did you encounter the download problem since you solved the problem with the BSOD?

Resuming partial downloads is only necessary if something unexpected (like a BSOD) happens during download. It 's also a little hard to reproduce and track down if there is a bug in the code. If you still have the problem and we can somehow reproduce it on a dev system we can try to fix it.


I can probably reproduce it by hard shutting down the PC during the download, but so could you ;) It just requires a project with a very large download (100MB+)

If it helps you reproduce it, it might be related to having Windows write-cache buffer flushing disabled on your drive? It's an SSD so I keep that disabled (technically enabled, but the wording of the option is iffy).



Would BOINC be affected by this?
ID: 53459 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 321
Germany
Message 53464 - Posted: 1 Apr 2014, 16:17:15 UTC

If your PC is shutting down hard there is a bigger problem than downloading from the beginning. This function is designed for a more frequent case with a loose internet connection. This way the Client has the time to save the file on disk if a loss of connection is detected (at least that's what it should do).

From what I see in the picture I'm not sure if this function is turned on or off but it would definitely have an impact on BOINC or any other application that writes on your SSD.

Is it possible that there are two Write-Caching policies? One Hardware related and one Windows related? I don't use SSDs so I can't say anything about the effects of this function or the settings.
ID: 53464 · Report as offensive

Message boards : BOINC client : BOINC doesn't write in-progress downloads to disk?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.