Message boards : Questions and problems : Use HDD for BOINC work files instead of SSD
Message board moderation
Previous · 1 · 2
Author | Message |
---|---|
Send message Joined: 19 Jan 14 Posts: 79 |
You don't judge by load. You look in logs. That makes sense, but I'm afraid that I don't know where these logs would be. Second, #8 provided then #5, #6, #7 are completely unnecessary. If you mean that step 8 makes steps 5, 6 and 7 unnecessary: Step 5: This is the punchline of the whole thing, which is to move the BOINC working directory from my SSD to my HDD. Step 6: That was just in preparation for creating the link. Step 7: I created the link as a result of Message 53626, and BOINC Manager started running again, so I believe it to be necessary. If that's not what you mean, then I don't understand that sentence. |
Send message Joined: 19 Jan 14 Posts: 79 |
If it still doesn't work run the following: The result of that is 27-Apr-2014 11:20:12 [---] Starting BOINC client version 7.2.42 for x86_64-pc-linux-gnu 27-Apr-2014 11:20:12 [---] log flags: file_xfer, sched_ops, task 27-Apr-2014 11:20:12 [---] Libraries: libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3 27-Apr-2014 11:20:12 [---] Data directory: /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client 27-Apr-2014 11:20:12 [---] No usable GPUs found 27-Apr-2014 11:20:12 [---] Host name: W2600CR 27-Apr-2014 11:20:12 [---] Processor: 32 GenuineIntel Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz [Family 6 Model 45 Stepping 7] 27-Apr-2014 11:20:12 [---] Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid 27-Apr-2014 11:20:12 [---] OS: Linux: 3.13.0-24-generic 27-Apr-2014 11:20:12 [---] Memory: 31.32 GB physical, 31.92 GB virtual 27-Apr-2014 11:20:12 [---] Disk: 916.77 GB total, 441.61 GB free 27-Apr-2014 11:20:12 [---] Local time is UTC +9 hours 27-Apr-2014 11:20:12 [---] VirtualBox version: 4.3.10_Ubuntur93012 27-Apr-2014 11:20:12 [---] Config: GUI RPCs allowed from: 27-Apr-2014 11:20:12 [pogs] URL http://pogs.theskynet.org/pogs/; Computer ID 34098; resource share 100 27-Apr-2014 11:20:12 [---] General prefs: from http://lhcathome2.cern.ch/test4theory/ (last modified 17-Nov-2013 17:08:05) 27-Apr-2014 11:20:12 [---] Host location: none 27-Apr-2014 11:20:12 [---] General prefs: using your defaults 27-Apr-2014 11:20:12 [---] Reading preferences override file 27-Apr-2014 11:20:12 [---] Preferences: 27-Apr-2014 11:20:12 [---] max memory usage when active: 16036.07MB 27-Apr-2014 11:20:12 [---] max memory usage when idle: 28864.93MB 27-Apr-2014 11:20:12 [---] max disk usage: 441.54GB 27-Apr-2014 11:20:12 [---] suspend work if non-BOINC CPU load exceeds 25% 27-Apr-2014 11:20:12 [---] (to change preferences, visit a project web site or select Preferences in the Manager) dir_open: Could not open directory 'slots' from '/media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client'. 27-Apr-2014 11:20:12 [---] gui_rpc_auth.cfg is empty - no GUI RPC password protection 27-Apr-2014 11:20:12 [---] Not using a proxy 27-Apr-2014 11:20:12 Initialization completed That Could not open directory 'slots' from '/media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client'is pretty suspicious. I've seen it several times before. I don't understand it because there's no directory 'slots' in /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client. |
Send message Joined: 5 Oct 06 Posts: 5131 |
ThatCould not open directory 'slots' from '/media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client'is pretty suspicious. I've seen it several times before. I don't understand it because there's no directory 'slots' in /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client. BOINC operates with three sub-directories below the main data directory. You have the main directory listed there: 27-Apr-2014 11:20:12 [---] Data directory: /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client so create these three directories within it: notices projects slots if they're not already there. Make sure that BOINC has permission to read and write to all three sub-directories, and in the case of 'projects' and 'slots' to execute files - including files in daughter directories. |
Send message Joined: 20 Nov 12 Posts: 801 |
el_gallo_azul@W2600CR:~$ ps -A|grep boinc 7372 ? 00:00:10 boincmgr It just means boincmgr doesn't have its input and output connected to any terminal. Was boinc supposed to be running at that time? If you start the service does it run for a moment and then crash like it did in the other thread? You don't judge by load. You look in logs. Well, kern.log and syslog in/var/log/ in case of crashes (these log names vary from one system to another) and std*.txt in BOINC's data directory in case BOINC doesn't do what you want it to do. Step 5: This is the punchline of the whole thing, which is to move the BOINC working directory from my SSD to my HDD. Technically, you could skip the move and just tell BOINC to use the new location. For BOINC this would look like a new computer. Of course if you had any workunits on board BOINC wouldn't know about them anymore. The top level directory is still something you need to create yourself, BOINC shouldn't have permissions to do it by itself. The result of that is And this time the client didn't crash? I can't see anything weird in the messages. ThatCould not open directory 'slots' from '/media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client'is pretty suspicious. I've seen it several times before. I don't understand it because there's no directory 'slots' in /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client. That error message is Debian add-on. I think it's a false alarm this time. The client is probably just looking if there is any slot directories to clean up. You can still make the directory yourself. Maybe that will make BOINC happy. Moving the data directory should just work. I don't have any good ideas at the moment so I think I need to try the Ubuntu package myself but that will be later next week. |
Send message Joined: 19 Jan 14 Posts: 79 |
BOINC operates with three sub-directories below the main data directory. You have the main directory listed there: I had checked that BOINC was working, just before moving /var/lib/boinc-clientto /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client, and the directory (which I have not changed in any way) contains the following subdirectories: notices projects .nv I'm happy to create a slots subdirectory, bearing in mind that BOINC was working OK before I moved this directory. In order to speed this up, it would be helpful if someone could give me the correct command to create the directory and set the correct permissions. Make sure that BOINC has permission to read and write to all three sub-directories, and in the case of 'projects' and 'slots' to execute files - including files in daughter directories. The three subdirectories have the following permissions which I have not changed: notices Owner: BOINC core client Access: Create and delete files Group: boinc Access: Create and delete files projects Owner: BOINC core client Access: Create and delete files Group: boinc Access: Create and delete files .nv Owner: BOINC core client Access: Create and delete files Group: boinc Access: None |
Send message Joined: 19 Jan 14 Posts: 79 |
It just means boincmgr doesn't have its input and output connected to any terminal. Nope I don't know what that means (the "connected" part). Was boinc supposed to be running at that time? If you start the service does it run for a moment and then crash like it did in the other thread? Just now: cd /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client/ && sudo -u boinc boinc followed by ps -A|grep boinc in another terminal gives: 5031 pts/2 00:00:00 boinc 5036 pts/2 00:00:00 boinc Technically, you could skip the move and just tell BOINC to use the new location. For BOINC this would look like a new computer. Yes but I didn't want to have to create subdirectories and permissions, hence the move. The top level directory is still something you need to create yourself, BOINC shouldn't have permissions to do it by itself. Which top-level directory? And this time the client didn't crash? I can't see anything weird in the messages. Yes if I: 1. Start BOINC Manager; then 2. Try to add a project (since my projects have disappeared), after about 10 seconds I get the message "The BOINC client has exited unexpectedly 3 times within the last 2 minutes. Would you like to restart it again?". Before I moved the directory "boinc-client", I added a project and BOINC all worked, downloading and starting on about 40 or 50 tasks before I paused it and moved the directory. Then I did the three things from this post: 1. Move (or create) the "boinc-client" directory; and 2. Create a symbolic link "var/lib/boinc-client" to the new "boinc-client" directory; and 3. Edit "/etc/default/boinc-client" to state the location of "boinc-client" and now it: 1. Doesn't show any projects or tasks; and 2. Doesn't let me add any projects. Moving the data directory should just work. Yes I agree with this. I am sure that there is some way to use the HDD instead of the SSD for BOINC working files, but so far I haven't been able to achieve it or even to find out how. |
Send message Joined: 20 Nov 12 Posts: 801 |
Been playing with the client for a while now. I still don't know what happened to your slots directory during the move. Either the whole data directory should have been moved or the files and directories that couldn't be moved should have been left at the original location. But since you could create the link it's like some of the files just vanished into thin air??? Anyway. Have you noticed the text in the lower right corner of the Manager? You know, the text that says "Connected" or "Connecting" or "Disconnected" or something like that. When you say your projects and tasks have disappeared has that text in the Manager ever said "Connected"? And has it said "Connected" before you tried to add a project? Next, I would like you to list the owner and permissions of all the directories in the data directory path, starting from /media. And not from file manager but from command line with "ls -ld /media" and so on. Please just copy-paste. You said you set your HDD to mount automatically at OS start. How and where did you set it? What's in your /etc/fstab? Btw, with the link in place you don't really need to edit the defaults file. Everything already goes through the link to the new location. |
Send message Joined: 19 Jan 14 Posts: 79 |
Since my last post, I have got BOINC running again as before, on my SSD (as the default installation configuration). I would rather be running BOINC now in the shorter term than spending loads of time trying to work out how to be able to run BOINC in the longer term. Hopefully it won't take too long to get a new SSD when I need one. I see that there is a directory "slots" in /var/lib/boinc-client. That was not there after I moved the directory "boinc-client" using sudo mv /var/lib/boinc-client/ /media/sdb1/var-lib-boinc-client/as described in Message 53609. I will have another go at getting it running on the HDD though, so that it doesn't get the big interruption when it's time to change the SSD.
I have seen it before, but I didn't notice what it said when I was trying to run BOINC on my HDD. That's a shame isn't it.
I'll do this, but I'll let BOINC run for a couple of weeks before I have another go.
I don't really understand the question (the "set it" part). This is the first time that I've had more than one drive in a computer. When I bought the computer, the HDD was set up to list as a directory "storage" within / and not have a disk mounted discretely, but I changed it, and now it looks the same straight after booting as an external disk does when plugged in.
Nothing special: # /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # <file system> <mount point> <type> <options> <dump> <pass> # / was on /dev/sda1 during installation UUID=21e4cb98-d3af-4a9a-9cdf-aa5e92493ed8 / ext4 errors=remount-ro 0 1 # swap was on /dev/sda5 during installation UUID=265ae6bf-eb9e-4100-98d4-b1fbec0b658d none swap sw 0 0 |
Send message Joined: 20 Nov 12 Posts: 801 |
Since my last post, I have got BOINC running again as before, on my SSD (as the default installation configuration). I would rather be running BOINC now in the shorter term than spending loads of time trying to work out how to be able to run BOINC in the longer term. That is quite understandable. :) I see that there is a directory "slots" in /var/lib/boinc-client. That was not there after I moved the directory "boinc-client" usingsudo mv /var/lib/boinc-client/ /media/sdb1/var-lib-boinc-client/as described in Message 53609. Yep. I tried that and tried it again while BOINC was still running but for me the slots directory didn't disappear. Not that it matters all that much, you'll just lose any half completed work and possibly entire cache, but I don't think the missing slots directory prevented BOINC from running. I have seen it before, but I didn't notice what it said when I was trying to run BOINC on my HDD. That's a shame isn't it. Yeah, the Manager doesn't make it easy to notice. It really could disable the menu entries when it's not connected.
Edit a file, click some buttons in some GUI tool, etc. This is the first time that I've had more than one drive in a computer. When I bought the computer, the HDD was set up to list as a directory "storage" within / and not have a disk mounted discretely, but I changed it, and now it looks the same straight after booting as an external disk does when plugged in. Based on the lack of "Storage" in fstab and your story I'm going to guess: You boot up the computer, open file manager and there is "Storage" listed under Devices. When you click the "Storage" an eject like icon appears next to it. Guessing some more: When you changed how the HDD showed in file manager, you either removed the "Storage" from fstab or went through Disks->your HDD->partition in it->Edit Mount Options->Automatic Mount Options->On. There is two problems with this. First, the HDD isn't mounted automatically when the OS starts. And second, when you let GNOME manage the mounting process the permissions gets set too tight so BOINC can't change it's working directory to the new data directory. When you are ready to try again: - Open the Disks programs and select the mount options - Switch off the automatic options - Select "Mount at startup" - Don't select "Show in user interface" (you want it but it doesn't work) - In mount options put something like "defaults,comment=x-gvfs-show" (in Mint 15 that box doesn't have label, it's the one between Icon Name and Mount Point) - Pick a mount point and type it in Mount Point (/storage or /mnt/storage is good, /media/el_gallo_azul/... isn't, let GNOME have that directory) - For Identify As UUID or LABEL based is good - Set Filesystem Type as auto or ext4 The "comment=x-gvfs-show" should make the "Storage" appear in file manager. At least it does so in Mint 15... |
Send message Joined: 6 Dec 06 Posts: 118 |
How about creating a 30 or 40GB partition on the HDD labeled /var and make sure it mounts at startup? That way, when installing BOINC from repositories it will automatically create the data directory on the HDD. |
Send message Joined: 19 Jan 14 Posts: 79 |
Yes When you changed how the HDD showed in file manager, you either removed the "Storage" from fstab or went through Disks->your HDD->partition in it->Edit Mount Options->Automatic Mount Options->On. No, I used the CLI, but I can't remember now what the command line said. |
Send message Joined: 19 Jan 14 Posts: 79 |
I have a funny feeling deep down inside that the problem is probably related to First, the HDD isn't mounted automatically when the OS starts. And second, when you let GNOME manage the mounting process the permissions gets set too tight so BOINC can't change it's working directory to the new data directory. despite the fact that the permissions for /.../boinc-client and its subfolders looked OK when I looked at them after moving the folder from /var/lib/boinc-client on the SSD to /media/el_gallo_azul/Storage/var-lib-boinc-client/boinc-client on the HDD. The other possibility is something to do with the fact that the folder /var/lib/boinc-client/slots exists now while BOINC is working on the SSD, but when I moved the folder /var/lib/boinc-client from the SSD to the HDD, all of a sudden there was no /.../boinc-client/slots. |
Send message Joined: 20 Nov 12 Posts: 801 |
Yes. After a reboot BOINC can't start because /media/el_gallo_azul/Storage isn't mounted. After you manually mount it BOINC still can't start because one of the directories have too tight permissions, probably /media/el_gallo_azul but that's not necessarily the only one. I forgot to mention earlier. That .nv directory in /var/lib/boinc-client is NVIDIA's OpenGL shader cache. I can imagine that it's used to cache CUDA or OpenCL kernels too. NVIDIA drivers puts that directory in user's home directory. For user boinc, that is /var/lib/boinc-client. So you'll want to keep that directory around either as a real directory or as a symlink to the new location or change boinc's home directory to the new location. |
Send message Joined: 19 Jan 14 Posts: 79 |
So you'll want to keep that directory around either as a real directory or as a symlink to the new location or change boinc's home directory to the new location. Do you mean to do something else once I've moved it? Also, there is no "boinc's home directory" specified in /etc/default/boinc-client. |
Send message Joined: 20 Nov 12 Posts: 801 |
So you'll want to keep that directory around either as a real directory or as a symlink to the new location or change boinc's home directory to the new location. Making a symlink old->new works ok, like you've done so far to make Manager work. I just figured I'd mention it. Also, there is no "boinc's home directory" specified in /etc/default/boinc-client. It's a user account thing so it's in /etc/passwd, modified with usermod. |
Send message Joined: 19 Jan 14 Posts: 79 |
After all that, I think I'll just leave it as is now. According to the Disks utility, the number of bad sectors hasn't increased from 302 for two weeks or more, and I have been running BOINC during that time. If my SSD is going to hold for a while, I'd rather than just spend the money (and the time) to replace it when the time comes than spend more time now trying to get BOINC to work nicely on my HDD. Thanks for your help. |
Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.2 or any later version published by the Free Software Foundation.