Message boards : Questions and problems : Mounting subdirectories blows BOINC's mind
Message board moderation
Author | Message |
---|---|
Send message Joined: 14 Oct 08 Posts: 15 ![]() |
Mounting a subdirectory, such as slots, on a different disk to the BOINC data directory results in disk space calculations that are incorrect. It would appear that the client checks the disk usage overall, and compares this to the available disk space of the data directory. I bumped my head on this over at this Rosetta@Home thread. It may be that this is intentional, but if it isn't then I thought I'd shout out, just in case anyone has looked into this. This method of checking disk free space is, of course, incorrect, but it may simply be too much of an unusual situation for anyone to really care. |
Send message Joined: 14 Oct 08 Posts: 15 ![]() |
Okay, so this does have a pretty annoying consequence. BOINC looks at the size of the disk (20 GB), and looks at the size of its data directory (18 GB). This 18 GB I got down to 2 GB thanks to compression and deduplication. Very nice. Unfortunately BOINC simply sees the 18 GB and so believes there is only 2 GB remaining, which of course there isn't. BOINC will never try to exceed 20 GB of stored data. This is disappointing. Indeed, there is a general problem with how BOINC looks at free space. My guess is that it can be tricked with overprovisioning, but we'll see how things go. |
![]() ![]() Send message Joined: 17 Nov 16 Posts: 904 ![]() |
All slots are contained in the BOINC data directory. If you have space constraints, just move the entire data directory. Not the slots. https://boinc.berkeley.edu/forum_thread.php?id=13664#98166 |
Send message Joined: 14 Oct 08 Posts: 15 ![]() |
That's what I ended up doing. My second post is still relevant to this setup though. I don't know how BOINC can look at the actual disk space used, though there are userspace tools do so, so it might be possible. |
![]() ![]() Send message Joined: 17 Nov 16 Posts: 904 ![]() |
The client pulls the info from the host in the host_info module for various platforms. int HOST_INFO::get_host_info(bool init) { int retval = get_filesystem_info(d_total, d_free); if (retval) { msg_printf(0, MSG_INTERNAL_ERROR, "get_filesystem_info() failed: %s", boincerror(retval) |
Send message Joined: 14 Oct 08 Posts: 15 ![]() |
total_space = (double)fs_info.f_frsize * (double)fs_info.f_blocks; free_space = (double)fs_info.f_frsize * (double)fs_info.f_bavail; Well this is clearly correct and gives correct values. $ stat -f /var/lib/boinc-client File: "/var/lib/boinc-client" ID: 165a0cf0f966f308 Namelen: 255 Type: btrfs Block size: 4096 Fundamental block size: 4096 Blocks: Total: 2621440 Free: 2310694 Available: 2239902 Inodes: Total: 0 Free: 0 get_filesystem_info will return the correct values. So something else is going on. I see in cs_scheduler.cpp that get_disk_usages() is called: int CLIENT_STATE::get_disk_usages() { unsigned int i; double size; PROJECT* p; int retval; char buf[MAXPATHLEN]; client_disk_usage = 0; total_disk_usage = 0; for (i=0; i<projects.size(); i++) { p = projects[i]; p->disk_usage = 0; retval = dir_size(p->project_dir(), size); if (!retval) p->disk_usage = size; } for (i=0; i<active_tasks.active_tasks.size(); i++) { ACTIVE_TASK* atp = active_tasks.active_tasks[i]; get_slot_dir(atp->slot, buf, sizeof(buf)); retval = dir_size(buf, size); if (retval) continue; atp->wup->project->disk_usage += size; } for (i=0; i<projects.size(); i++) { p = projects[i]; total_disk_usage += p->disk_usage; } retval = dir_size(".", size, false); if (!retval) { client_disk_usage = size; total_disk_usage += size; } return 0; } Here we can see that dir_size is called, which does as you might expect. $ sudo du -sh /var/lib/boinc-client 1.8G /var/lib/boinc-client $ df -h /var/lib/boinc-client Filesystem Size Used Avail Use% Mounted on /dev/sdb 10G 1.2G 8.6G 13% /var/lib/boinc-client So, on calculating the amount of disk space used so that the information can be sent to the server, the client simply adds up the amount of data in the relevant directories. This process is unaware of TRUE disk use (i.e. after any deduplication and/or compression). The two numbers above show how this information varies. BOINC looks at the size of the disk (20 GB), and looks at the size of its data directory (18 GB). Please correct me if I'm wrong. :-) Note: The disk size and amount of stored data differs from when I first looked at this, but the result can still be seen. |
![]() ![]() Send message Joined: 17 Nov 16 Posts: 904 ![]() |
Please correct me if I'm wrong. :-) No, I don't think you are wrong. BOINC was never written to consider any file or disk compression, and has issues with any file system other than legacy DOS or Windows based. Code from 2002 is still predominant in the codebase. BOINC needs to be rewritten from the ground up for better considerations of modern OS' and hardware. I think that trying to continue to support legacy OS's and hardware from 18 years ago is a fools errand. Better to cut them off and accept current realities. |
Send message Joined: 14 Oct 08 Posts: 15 ![]() |
Interesting to see that Win 9x isn't listed as supported, but that a lot of projects still support Win 9x. That's fine, but if you're running old OS then you're running an old BOINC anyway. And it will always be available. Saying that, disk compression was definitely available during the Win 9x and DOS era (Doublespace, etc.), though I see the code is different for Windows and I haven't taken much notice of it. Looks like a very old issue that won't go away on its own. |
Copyright © 2025 University of California.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.2 or any later version published by the Free Software Foundation.