Can I reduce I/O using memory?

Message boards : Questions and problems : Can I reduce I/O using memory?
Message board moderation

To post messages, you must log in.

AuthorMessage
somitcw

Send message
Joined: 12 Jan 17
Posts: 4
Message 75222 - Posted: 12 Jan 2017, 17:21:18 UTC

BOINC 7.6.33 (x64)
current Windows 10
probably all projects but Asteroids@home is what I'm running today.

Windows Task Manager, Performance tab, Open Resource Manager
shows slow constant I/O writing to BOINC's ProgramData folder.
Files are sub-miniature tiny measured in a few KB but always busy.
But BOINC and the System are using the files constantly.

Disk drives and USB jump drives wear out faster with constant access.
USB jump drives wear out faster with writing than if BOINC was reading.

With many recommending Windows 10 system have a minimum of 2GB of
memory but many using many times that, is there a way to tell BOINC to
keep most data in memory?

When I have at least 2GB of memory, 2KB for a disk file is one millionth of that.
Memory is also faster that a USB jump drive.

BOINC writing once a minute, once per blue moon, or only on
normal shutdown would not be an issue.
Is there a BOINC option to buffer the write or software?
Could software like VirualBox do that?

P.S. I installed BOINC this time on a USB jump drive because years ago
it was beating my system hard drive to death.
ID: 75222 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 75300 - Posted: 13 Jan 2017, 18:45:20 UTC - in response to Message 75222.  

There's a few projects that have apps that really write a lot to the disk. Some people running those projects have BOINC keep its data directory in ramdrive. That has the obvious drawback of the data directory being wiped clean every time you restart the system.

Setting checkpoint interval higher can reduce the amount of writes somewhat.

I don't think running BOINC will kill any HDD and probably not any modern SSD, USB flash drives maybe.
ID: 75300 · Report as offensive
somitcw

Send message
Joined: 12 Jan 17
Posts: 4
Message 75333 - Posted: 16 Jan 2017, 21:23:30 UTC - in response to Message 75300.  

I have looked into your RamDisk suggestion but haven't found
one that I liked yet. Most don't say if data is persistent so
this could end up a long trial and error venture.

With BOINC pounding on hard-drives 24/7, it is nice to know
that you declare "I don't think running BOINC will kill any HDD".
That is what you said.
ID: 75333 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 75342 - Posted: 17 Jan 2017, 21:13:09 UTC - in response to Message 75333.  

HDDs are perfectly capable of failing even without BOINC's help. They have always had varying lifetimes. Some run for a decade or two and others wear out their bearings after only a few thousands of hours. It will be pretty hard to pinpoint a HDD's failure to any specific program.

Here's my own experiences on running BOINC on HDDs.

HDD 1: bought some 15 years ago, power on hours = 81395 = 9 years 3 months, used for BOINC data for about 5 years, currently storage for DVR with about 2-3 GB per day writes
HDD 2: bought some 10 years ago, power on hours = 35398 = 4 years, used for BOINC data for about 4 years, still used for BOINC data

HDD 2 is one of those Seagates that originally had buggy firmware. Power on hours was probably reset when I de-bricked it. I don't have any failed HDDs.

And now I have been using SSD for BOINC data for about three weeks. Come back in a couple years time and ask me what I then think of that.

Btw. Windows' Resource Monitor's values are averages over one minute of data.
ID: 75342 · Report as offensive
Profile Agentb
Avatar

Send message
Joined: 30 May 15
Posts: 265
United Kingdom
Message 75347 - Posted: 17 Jan 2017, 23:11:21 UTC - in response to Message 75333.  
Last modified: 17 Jan 2017, 23:13:24 UTC

somitcw wrote:
With BOINC pounding on hard-drives 24/7, it is nice to know
that you declare "I don't think running BOINC will kill any HDD".
That is what you said.


And I would concur with Juha. I have a PC pounding away 7x24 BIONC for 5 years 8 months and 19 days daily backups etc.

HDD SMART statistics reveal it is about 70% of it's reallocated sector allocation which suggests I have another two years left before I need to worry about anything.

Another PC has a small SSD pounding silently away for nearly two years at BOINC , while running hot double GPU system, it has a has wear level indicator at 50%. If the press is to believed, this is just an indicator and the life is typically another 30-100+% longer - I'll leave it to the reader to search.

If you are really worried about this, monitor the SMART drive stats monthly, and then make comment.
ID: 75347 · Report as offensive
somitcw

Send message
Joined: 12 Jan 17
Posts: 4
Message 75360 - Posted: 18 Jan 2017, 14:16:34 UTC - in response to Message 75347.  

Boinc on my USB jump drive died.
So far, I haven't even been able to uninstall Boinc.
I assUme that the flaky USB controller contributed more to
the failure than the constant use.

While others may want to add pounding to a boot disk, I don't.
After I find suitable RamDisk software, I'll reinstall Boinc.
ID: 75360 · Report as offensive
Juha
Volunteer developer
Volunteer tester
Help desk expert

Send message
Joined: 20 Nov 12
Posts: 801
Finland
Message 75372 - Posted: 18 Jan 2017, 20:32:01 UTC - in response to Message 75360.  

Fair enough.

Using ramdrive may result in lots of cursing and tearing your hair out. As an alternative, you could try sourcing a stack of second-hand low capacity HDDs from, say, nearby recycling station or online flea market. Many BOINC projects don't need much disk space. An HDD with capacity of a few gigabytes would be enough for many and for some even a few hundred megabytes is enough. As long as you don't pay anything for the disks it doesn't matter even if some of them die. I would hope nobody is crazy enough to actually ask any money for such disks.
ID: 75372 · Report as offensive
tanstaafl9999

Send message
Joined: 28 Feb 12
Posts: 23
United States
Message 75402 - Posted: 20 Jan 2017, 1:21:52 UTC

I used to have constant, very heavy hard drive activity on my computer, which I tracked back to BOINC a couple of years ago.

I searched these forums and found a thread about the problem: the suggested solution was to increase the checkpoint time to 900 seconds. I did that, and the problem is solved.

I don't remember the original (default) value for the checkpoint time, but it was enough to pound on the hard drive non-stop with six to ten CPU cores all running BOINC tasks. I have a feeling the default checkpoint time was set years ago when CPUs had fewer cores and weren't running enough BOINC tasks simultaneously to give the hard drive a workout even with frequent checkpointing.

Note that with a longer time between checkpoints, you will lose more unsaved data when you reboot or shut down the computer or otherwise stop BOINC from running, To me, that's not an issue: I only reboot or shutdown the computer every few days, and losing a few minutes of calculations a couple of times a week is not a problem for me.

So before you get involved with a RAM disk, I would suggest increasing the checkpoint time to 900 seconds (or whatever you're comfortable with) to see if that helps.
ID: 75402 · Report as offensive
Vort

Send message
Joined: 31 Dec 16
Posts: 3
Bulgaria
Message 75403 - Posted: 20 Jan 2017, 7:16:25 UTC - in response to Message 75402.  
Last modified: 20 Jan 2017, 7:53:32 UTC

...
I would suggest increasing the checkpoint time to 900 seconds (or whatever you're comfortable with) to see if that helps.
What if you set the checkpoint time to 24 hours?

I am running Ubuntu server 24/7 on a Single board computer that uses an SD card as a hard drive. I don't halt it at all, just reboot once a month or two.

Some years ago I installed and run i2p https://en.wikipedia.org/wiki/I2P on it.
After installing on the SD card I copied its directories to RAM (to /var/run/ which is tmpfs https://en.wikipedia.org/wiki/Tmpfs) and redirected the input/output operations to them by a symbolic link.
This way i2p ran totally in RAM, even the symbolic link was read from there ;). Once a day or two I manually copied these working directories to the SD card after suspending the program.

Had no problems.
ID: 75403 · Report as offensive
HAL9000
Help desk expert
Avatar

Send message
Joined: 13 Jun 14
Posts: 81
United States
Message 75593 - Posted: 31 Jan 2017, 23:26:07 UTC

I have for various reasons needed to use a RAM disk with BOINC over the years. My preference is the free version from dataram.
http://memory.dataram.com/products-and-services/software/ramdisk
It gives you the option to save the ramdisk to an image and load it upon start. Also you can set how often you want it to write the ramdisk to the image.
Besides writing at system shutdown I would set it to write about once per hour. So if the system crashed and wasn't about to write to the image I would only lose 1hr of crunching.

There may be better free options, but I've been using their software for many years.
ID: 75593 · Report as offensive

Message boards : Questions and problems : Can I reduce I/O using memory?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.