Thread 'Old way doesn't work -- how do I change where BOINC looks for its data directory (Kubuntu 22.04)?'

Message boards : Questions and problems : Old way doesn't work -- how do I change where BOINC looks for its data directory (Kubuntu 22.04)?
Message board moderation

To post messages, you must log in.

AuthorMessage
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111545 - Posted: 12 Apr 2023, 22:46:10 UTC

I used to be able to just move the boinc-client folder and update the link named boinc and BOINC manager and clients would happily find the folder where I put it (somewhere that has more space than my / volume). The new manager, however, is intent on keeping my system so secure I'm not actually allowed to run it (well, more or less); I even have to enter a password when it auto-starts after a system restart.

I tried the above on a freshly installed BOINC (boinc-manager, boinc-client, boinc-client-nvidia-cuda, and their dependencies) and when I launched the manager, I got an error saying "The file was not found" -- saying nothing about which file, of course.

Would I be correct to assume I now have to go through some arcane set of steps, potentially involving an animal sacrifice, to get BOINC to store its data on a volume with more free space? And while I'm asking, why isn't this an option in the BOINC manager? Seems like it would be pretty simple to include in the preferences, something like "BOINC Data Folder" and a type-in or browse option (could even look pretty much the same for Windows, Mac, or Linux, but using the system default file selector).
ID: 111545 · Report as offensive
computezrmle

Send message
Joined: 2 Feb 22
Posts: 84
Germany
Message 111547 - Posted: 13 Apr 2023, 5:18:15 UTC - in response to Message 111545.  

I used to be able to just move the boinc-client folder...

You may check whether you accidentally changed ownership/permissions during this move.
Use a tool that preserves both.

After the move a directory symlink oldlocation -> newlocation should do the trick.
Most Linux systems do so, e.g. for /bin -> /usr/bin.
ID: 111547 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111551 - Posted: 13 Apr 2023, 21:32:42 UTC - in response to Message 111547.  

Ugh. I'm trying to move the folder to an ntfs volume and AFAIK those don't support ownership and permissions as Linux understands them.

I do have a couple folders on ntfs volumes mounted as folders inside my /home; would that work in /var/lib for this, or would I have the same problem with losing ownership?

Obviously, the long term problem is that my storage is getting too close to full; I've got a bunch of volumes with a few GB to 20-25 GB free (the ones with more are all old legacy folders from Windows XP). but my SSD is too full. Solution will be to buy another SSD, move my /home to the new drive, and expand the / volume into the freed space. Which, the way projects like this usually go, will take a weekend when I can spare $80-$100 for a new SSD.
ID: 111551 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2725
United Kingdom
Message 111553 - Posted: 14 Apr 2023, 4:39:11 UTC - in response to Message 111551.  

Ugh. I'm trying to move the folder to an ntfs volume and AFAIK those don't support ownership and permissions as Linux understands them.

I do have a couple folders on ntfs volumes mounted as folders inside my /home; would that work in /var/lib for this, or would I have the same problem with losing ownership?

Obviously, the long term problem is that my storage is getting too close to full; I've got a bunch of volumes with a few GB to 20-25 GB free (the ones with more are all old legacy folders from Windows XP). but my SSD is too full. Solution will be to buy another SSD, move my /home to the new drive, and expand the / volume into the freed space. Which, the way projects like this usually go, will take a weekend when I can spare $80-$100 for a new SSD.


You could set the new drive up with a partition /var/lib/boinc-client As long as you don't run any projects that crash leaving their files behind, that way you don't have to worry about non-boinc stuff impinging on the space.
ID: 111553 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111554 - Posted: 14 Apr 2023, 11:46:18 UTC - in response to Message 111553.  



You could set the new drive up with a partition /var/lib/boinc-client As long as you don't run any projects that crash leaving their files behind, that way you don't have to worry about non-boinc stuff impinging on the space.


That's probably the sensible way to do it. Give BOINC ten or twenty GB to work in, and it'll never complain again -- and if it does, I can just clear out that partition and it'll go back to work.

And I hadn't realized how cheap SSDs had gotten -- I just got 1 TB and a three-pack of cables for under $60, same-day delivery from Amazon. Last time I bought a terabyte it was a platter drive and cost me $300 or so -- that's the drive that still has the Windows (98 and XP) legacy partitions on it. I should spend a few hours cleaning it up sometime...
ID: 111554 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111583 - Posted: 16 Apr 2023, 1:07:35 UTC
Last modified: 16 Apr 2023, 1:18:08 UTC

Okay, new 1 TB SSD installed (thanks, Amazon, for rapid free delivery!). partitioned, etc. BOINC now has 20 GB to play in. Looked up how to create symlinks across different devices (hint: Linux apparently likes to make commands Byzantine in scope, and then create obvious-seeming aliases that don't do what you'd expect them to) so the "boinc" shortcut to "boinc-client" finds another shortcut to an entire partition reserved for BOINC tasks.

I had to manually start the client (executing boinc in terminal) before BOINC Manager would let me connect to projects. And after creating the gui_rpc_auth.cfg file (touch gui_rpc_auth.cfg) and enduring the warning that any user on this machine can control BOINC (no problem, I lock the screen when I'm not at console to keep the cats from going on the internet, and they can't type anyway) -- but as soon as I restarted Kubuntu, I got the "can't start the client" popup (despite having been prompted for my password when the desktop loaded) in BOINC Manager. The client service isn't starting when Kubuntu starts up.

Edit: By way of a little hack in the autostart settings, I was able to get the client to start -- but now BOINC Manager is telling me "Invalid RPC password. Try reinstalling BOINC." Which I've done (sudo apt reinstall boinc) without change.
ID: 111583 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111585 - Posted: 16 Apr 2023, 10:52:56 UTC
Last modified: 16 Apr 2023, 11:46:51 UTC

Sigh...

I found the direct command to start boinc-client as a service (/etc/init.d/boinc-client which is a good sized shell script). It works, seemingly (status reports running), but starting BOINC Manager kills it, then reports "BOINC Manager (Pre-release) is not able to start a BOINC client. Please run the client and try again." I feel like I'm in a loop here.

I've replaced the shortcut method to point to the 20 GB data directory partition with editing /etc/boinc-client/config.properties -- which is apparently the "proper" way to redirect the data directory -- but this seems to have changed nothing.

I've been running @Home projects since before they were consolidated into BOINC, except for a couple years when I couldn't take the performance hit and couldn't afford a hardware upgrade. It used to be the case you'd just download the client and manager, install them, connect to your chosen projects, and done. CPU monitor then goes to 100% for all cores when not actively using the machine (at least after getting settings right) and no need to think about it unless something broke it. I have to say, I'm very disappointed with what's available now.

I've understood from a thread I found while looking for solutions to this that the Ubuntu packagers grabbed a "not ready" version of the manager and stuck it in their "fully tested" repositories. Is there a way to obtain a Linux version of BOINC without going through the Ubuntu package managers? A Snap or Flatpak? IMO, this is the way something like this should install, especially going forward after this incident with Ubuntu packagers.

EDIT: D'oh! BOINC Manager apparently is available as a Flatpak. Let me try installing that instead of fighting with this...

And that's not working very well, either. Installed and runs pretty well -- except BOINC Manager locks up at "Communicating with Project" after I enter my password to connect to either of my projects...

And it appears this is using my /home for its data directory space again, with no apparent way to change it (I found some hints that this may only be possible after connecting to/installing a project app).
ID: 111585 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2725
United Kingdom
Message 111586 - Posted: 16 Apr 2023, 11:28:37 UTC

I compile mine from source obtained from git-hub Sorting out the dependencies can take a while unless you are more conversant with them all than I am but certainly doable.
ID: 111586 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111587 - Posted: 16 Apr 2023, 11:52:26 UTC - in response to Message 111586.  
Last modified: 16 Apr 2023, 11:53:01 UTC

I compile mine from source obtained from git-hub Sorting out the dependencies can take a while unless you are more conversant with them all than I am but certainly doable.


I've been Debian-based only since 2011 and still haven't done a compile from source that wasn't in a package. Generally, if I have to do that, a software isn't really ready to work for users who aren't Linux wizards.

My expertise level is that of someone who can type recommended commands at a command line, make and remove files and directories and navigate the file system (though I prefer to navigate to a folder in Dolphin then "Open terminal here" if it's more than one or two levels up or down or the name of anything in the path will require quoting). Beyond that, I'm like an average Windows user who got fed up with Microsoft's methods back when XP was still supported: IOW, I can follow directions and usually find them on the web, but I don't even attempt to figure out an esoteric multi-option command line on my own.

Also, note an edit to my previous post, possibly while/after the above was posted.
ID: 111587 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111589 - Posted: 16 Apr 2023, 12:57:48 UTC - in response to Message 111587.  

Good grief, that's an annoying bug.

Apparently, the Flatpak BOINC Manager will lock up if one of two things happens (not certain which): your first attempt to connect to a project after installation is to one you're already a member of, or you enter an incorrect project password. I tried connecting to a new-to-me project, and it hooked right up; after doing so (and removing the project I wasn't interested in) and looking up the passwords in my browser's manager, I was able to connect to both Einstein and Milkyway and they're now happily waiting for my computer to be "unused" to run tasks.

Now I just need to figure out how to move the data directory for the flatpak to the 20 GB partition I set up for BOINC and I can ignore this again until next time I upgrade my Kubuntu.
ID: 111589 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2725
United Kingdom
Message 111590 - Posted: 16 Apr 2023, 13:17:04 UTC

The first time I compiled from source, my knowledge level was about what you describe. I think it took me about a week on and off to get it to work. The biggest headache was working out the packages that contain missing libraries. Next time I do it on a clean install I should really make a note of exactly what I need to add, then if nothing has changed, I should be able to do it all in one go!
ID: 111590 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111591 - Posted: 16 Apr 2023, 19:23:17 UTC - in response to Message 111590.  

if nothing has changed,


But you know that's a fantasy state. If you're having to build from source again, it's because you've upgraded your OS, most likely, and you'll also be getting a newer version of the BOINC Manager code. What you learned last year will only serve as guidelines for what you need to do next year.

As I tell people from time to time, I don't run Linux because installing and troubleshooting operating systems is a hobby for me, I run Linux because it's (mostly) FOSS, not from Microsoft, fully featured, supports almost every piece of hardware you can plug in, and it has better community support than Windows -- and I don't run/can't afford Apple hardware, so their OS is hard to get running (not to mention completely unsupported on a Hackintosh, and much worse for lock-in than Windows).

For instance, I don't like Discover, so one of the first things I do when I get a new Kubuntu install running is install Synaptic with sudo apt update && sudo apt install synaptic.

Anyway, my new SSD is installed, /etc/fstab updated to mount a 700+ GB partition as /home instead of the old 160 GB that only had a few GB free, and in that process, the contents of old home copied to new home via Gparted "copy partition" and "paste partition" -- and with all that free space, flatpak BOINC has half a gigabyte of free space and is once again happily churning away. And I was able to expand my / partition to give Kubuntu another 20 GB for its install space, increase my /swap to 32 GB (to match my RAM), give Timeshift 200 GB to store snapshots -- all because BOINC ran out of work space. Now I need to find an efficient way to move hundreds of files from my ~/Downloads into the freed space that was too full as my old /home so I can mount that partition as my ~/Downloads...
ID: 111591 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2725
United Kingdom
Message 111592 - Posted: 16 Apr 2023, 20:37:24 UTC

As I tell people from time to time, I don't run Linux because installing and troubleshooting operating systems is a hobby for me, I run Linux because it's (mostly) FOSS, not from Microsoft, fully featured, supports almost every piece of hardware you can plug in, and it has better community support than Windows --


Fair point. I first defenestrated to get away from Microsoft. Pretty sure if compiling from source had not got a lot quicker for me, (I can now do it half an hour including finding any missing dependencies) I would long ago have given up. BOINC is about the only thing I routinely compile from source. Occasionally I see something I want to try that I can only get that way but that is no more than two or three times a year.
ID: 111592 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111599 - Posted: 17 Apr 2023, 21:39:00 UTC - in response to Message 111592.  

Occasionally I see something I want to try that I can only get that way but that is no more than two or three times a year.


And in twelve years of running Linux exclusively (plus as dual-boot with Windows XP for a couple years before that), I haven't found one. Either I find a source of a compiled binary (*.tar.gz does not scare me, nor do non-repository .deb packages) or I find an alternative. Then again, I'm also not someone who installs new software every week just to try something new; I tend to find a tool that does the job and stick with it (hence why I still install Synaptic and Gparted as soon as I'm up and running on any new install -- the latter saved me when I installed 20.04, because there was a serious bug in the version of KDE Partition Manager that shipped with the 20.04.1 binary).
ID: 111599 · Report as offensive
Brian Nixon

Send message
Joined: 19 Apr 23
Posts: 16
United Kingdom
Message 111610 - Posted: 19 Apr 2023, 12:58:17 UTC - in response to Message 111545.  

why isn't this an option in the BOINC manager?

Because the location of the data directory is not decided by BOINC. It’s decided by however the client is installed/​invoked.

The data directory is simply the working directory in which the BOINC client is executed. On Linux systems that use the BOINC-supplied scripts, this is set by the variable BOINCDIR. On Windows, it’s in the Registry value HKLM\​SOFTWARE\​Space Sciences Laboratory, U.C. Berkeley\​BOINC Setup\​DATADIR.

(Also: if the directory were a client preference, there’d be a chicken-and-egg problem to resolve because for the Manager to be able to set client preferences, it needs to be connected to a running client. And if the client is running, its data directory has already been decided…)
ID: 111610 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111627 - Posted: 21 Apr 2023, 23:58:04 UTC - in response to Message 111610.  

Well, okay, and the standard software-developer solution for a chicken-and-egg problem is to do something arbitrary that makes perfect sense to a developer (who may or may not have two heads, because all that knowledge won't fit in one) but no sense at all to an end user who's just trying to accomplish what ought to be a fairly straightforward task.

But now, I've got an entirely different problem.

I was having some issues with update kernels not booting to GUI (in my case, KDE Plasma 5 desktop), which turned out to be due to dkms somehow having gotten uninstalled so the nVidia driver blob wasn't getting built into the new kernels. Working with the Kubuntu forums, I managed to get that fixed (someone posted a link to a one-liner that rebuilds all dkms drivers for all eligible kernels) and now I can boot freshly updated kernels again (I think -- haven't had a new kernel update since the fix).

BUT: since fixing that, Einstein@Home is reporting "missing GPU" for all tasks. I'm currently on Kubuntu 22.04.2, with kernel 5.19.0-40-generic (the HWE kernel path) and nVidia 530 (the recommended version for this Kubuntu an kernel) installed via ubuntu-drivers. Every test I know how to run says I have a fully functioning nVidia driver, I have the CUDA libraries installed, and have just installed nvidia-cuda-toolkit in order to check my CUDA version:

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0


I presume this is a supported CUDA version. This worked on nVidia 515, but I'd have to purge and reinstall nVidia drivers (possibly from command line) to get back to that version; Kubuntu's Driver manager won't even let me try.
ID: 111627 · Report as offensive
ProfileKeith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 893
United States
Message 111628 - Posted: 22 Apr 2023, 1:56:33 UTC - in response to Message 111627.  

Before worrying about what the OS says, what does BOINC say?

Does BOINC detect the card and CUDA and/or OpenCL drivers?

Boinc tells you what it has to work with in the first 30 lines of the Event Log startup. If you don't see any detected card(s) in the startup, you aren't going to be able to use a card for BOINC projects.

You should see something like this:

19-Apr-2023 10:06:52 [---] CUDA: NVIDIA GPU 0: NVIDIA GeForce RTX 2080 (driver version 525.99, CUDA version 12.0, compute capability 7.5, 7974MB, 7741MB available, 10598 GFLOPS peak)
19-Apr-2023 10:06:52 [---] CUDA: NVIDIA GPU 1: NVIDIA GeForce RTX 2080 (driver version 525.99, CUDA version 12.0, compute capability 7.5, 7972MB, 7507MB available, 10598 GFLOPS peak)
19-Apr-2023 10:06:52 [---] CUDA: NVIDIA GPU 2: NVIDIA GeForce RTX 2080 (driver version 525.99, CUDA version 12.0, compute capability 7.5, 7974MB, 7741MB available, 10598 GFLOPS peak)
19-Apr-2023 10:06:52 [---] OpenCL: NVIDIA GPU 0: NVIDIA GeForce RTX 2080 (driver version 525.105.17, device version OpenCL 3.0 CUDA, 7974MB, 7741MB available, 10598 GFLOPS peak)
19-Apr-2023 10:06:52 [---] OpenCL: NVIDIA GPU 1: NVIDIA GeForce RTX 2080 (driver version 525.105.17, device version OpenCL 3.0 CUDA, 7972MB, 7507MB available, 10598 GFLOPS peak)
19-Apr-2023 10:06:52 [---] OpenCL: NVIDIA GPU 2: NVIDIA GeForce RTX 2080 (driver version 525.105.17, device version OpenCL 3.0 CUDA, 7974MB, 7741MB available, 10598 GFLOPS peak)
ID: 111628 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111634 - Posted: 22 Apr 2023, 14:09:42 UTC - in response to Message 111628.  
Last modified: 22 Apr 2023, 14:19:45 UTC

I have the following:

Thu 20 Apr 2023 07:35:44 PM EDT | | No usable GPUs found
Thu 20 Apr 2023 07:35:44 PM EDT | | app version refers to missing GPU type NVIDIA
Thu 20 Apr 2023 07:35:44 PM EDT | Einstein@Home | Application uses missing NVIDIA GPU
Thu 20 Apr 2023 07:35:44 PM EDT | | app version refers to missing GPU type NVIDIA
Thu 20 Apr 2023 07:35:44 PM EDT | Einstein@Home | Application uses missing NVIDIA GPU
Thu 20 Apr 2023 07:35:44 PM EDT | | app version refers to missing GPU type NVIDIA
Thu 20 Apr 2023 07:35:44 PM EDT | Einstein@Home | Application uses missing NVIDIA GPU


Question is, why, and how do I fix it? The OS detects the hardware:

lspci | grep "NVIDIA"
01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)


The driver is installed and running (not sure why it shows 11 versions??):


ubuntu-drivers list
nvidia-driver-515
nvidia-driver-470
nvidia-driver-530
nvidia-driver-525
nvidia-driver-470-server
nvidia-driver-450-server
nvidia-driver-418-server
nvidia-driver-390
nvidia-driver-515-server
nvidia-driver-525-server
nvidia-driver-510


So, why wouldn't BOINC have detected the GPU?

EDIT: Okay, that was unexpected. I checked flatpak and found there was an update for BOINC; installing it, killing all the BOINC and project processes, and restarting BOINC has apparently fixed this issue. I now have Einstein happily running on 0.9 CPU + 1 GPU, and when BOINC started I had this:

Sat 22 Apr 2023 10:15:59 AM EDT | | CUDA: NVIDIA GPU 0: NVIDIA GeForce GTX 1070 (driver version 530.41, CUDA version 12.1, compute capability 6.1, 8106MB, 8106MB available, 6463 GFLOPS peak)
Sat 22 Apr 2023 10:15:59 AM EDT | | OpenCL: NVIDIA GPU 0: NVIDIA GeForce GTX 1070 (driver version 530.41.03, device version OpenCL 3.0 CUDA, 8106MB, 8106MB available, 6463 GFLOPS peak)
ID: 111634 · Report as offensive
ProfileKeith Myers
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 17 Nov 16
Posts: 893
United States
Message 111638 - Posted: 22 Apr 2023, 20:01:44 UTC

Generally it is a bad idea to install the CUDA libraries as they install their own driver outside of standard repo Nvidia drivers plus they set up their own development environment with paths that the stock Boinc client does not know how to use or expects to search to find the standard Nvidia driver resources.

You only need to install the CUDA dev kit if you expect to develop your own applications or use some of the enhanced features like machine or reinforcement learning.

The basic Nvidia runtime driver installation is sufficient to run any BOINC project app.
ID: 111638 · Report as offensive
Zeiss Ikon

Send message
Joined: 2 Jul 17
Posts: 36
United States
Message 111651 - Posted: 23 Apr 2023, 15:13:24 UTC - in response to Message 111638.  

I think we're talking about two different CUDA libraries. I don't have any nVidia development libraries -- but I do have two or three libraries that have description of "compute libraries for CUDA computation" and were automatically installed when ubuntu-drivers installed the upgrade from nvidia 515 to nvidia 530.
ID: 111651 · Report as offensive

Message boards : Questions and problems : Old way doesn't work -- how do I change where BOINC looks for its data directory (Kubuntu 22.04)?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.