Effect of settings on multi-core tasks.

Message boards : Questions and problems : Effect of settings on multi-core tasks.
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 110264 - Posted: 28 Oct 2022, 13:29:34 UTC

I have BOINC set to download 0.1days worth of work and up to 0 days worth of additional work. On LHC I am mostly getting multi-core tasks that run on 8 cores. (I have number of cores set to 50% max, i.e. half the 16 cores BOINC sees. It would seem that BOINC can not see that the tasks it is downloading are going to use 8 cores each as the total work downloaded amounts to 0.5 of a day.
ID: 110264 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110265 - Posted: 28 Oct 2022, 14:27:53 UTC - in response to Message 110264.  

Already reported at https://github.com/BOINC/boinc/issues/4151 (Jan 2021).

This one should be trivial to fix, if anyone can be bothered.
ID: 110265 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 110266 - Posted: 28 Oct 2022, 15:11:04 UTC - in response to Message 110265.  

Already reported at https://github.com/BOINC/boinc/issues/4151 (Jan 2021).

This one should be trivial to fix, if anyone can be bothered.
Thanks Richard, guess I should have looked there before posting. If more projects start doing multi-core work, then there may be some motivation to fix it. Main reason for raising it was the forthcoming multi-core stuff on CPDN as once it has been through testing and moves to the main site, there are sure to be people noticing it.
ID: 110266 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110278 - Posted: 31 Oct 2022, 22:16:20 UTC - in response to Message 110266.  

Looks like we have a taker: https://github.com/BOINC/boinc/pull/4992.

Now, how can we persuade MilkyWay or PrimeGrid to load it for testing?
ID: 110278 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 110282 - Posted: 1 Nov 2022, 5:17:35 UTC - in response to Message 110278.  

Looks like we have a taker: https://github.com/BOINC/boinc/pull/4992.

Now, how can we persuade MilkyWay or PrimeGrid to load it for testing?

Or LHC?
ID: 110282 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110283 - Posted: 1 Nov 2022, 8:31:17 UTC - in response to Message 110282.  

Or LHC?
I was going to say I hadn't seen any MT apps there, but I see they've now got a long_native_mt plan task. No work currently, but I'll ask Laurence.
ID: 110283 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 110284 - Posted: 1 Nov 2022, 10:32:28 UTC - in response to Message 110283.  

Or LHC?
I was going to say I hadn't seen any MT apps there, but I see they've now got a long_native_mt plan task. No work currently, but I'll ask Laurence.
I seem to get nearly all mt tasks from them. Interestingly I did an update with one task still running and unlike when I am out of work which leads to 8 being downloaded, I only got one.
ID: 110284 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110285 - Posted: 1 Nov 2022, 11:18:54 UTC - in response to Message 110284.  

OK, I've written to Laurence, and attached my Linux boxes. I'm trying to stay native - don't want to add VBox as well - and they may prefer to use their dev site. We'll see what happens.
ID: 110285 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110286 - Posted: 1 Nov 2022, 11:51:30 UTC - in response to Message 110285.  

And I've snagged three.

01/11/2022 11:41:36 | LHC@home | Requesting new tasks for CPU
01/11/2022 11:41:36 | LHC@home | [sched_op] CPU work request: 16034.67 seconds; 0.26 devices
01/11/2022 11:41:37 | LHC@home | [sched_op] estimated total CPU task duration: 23975 seconds
They are designated to run on all six cores, and estimated to run for 2 hours 13 minutes - I make that a total of 143,784 core-seconds.
ID: 110286 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 110287 - Posted: 1 Nov 2022, 12:19:52 UTC - in response to Message 110286.  

They are designated to run on all six cores, and estimated to run for 2 hours 13 minutes - I make that a total of 143,784 core-seconds.
The last Atlas task I ran was 34920core seconds running on 8 cores 1 hour 12 minutes 45 seconds.
ID: 110287 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110288 - Posted: 1 Nov 2022, 14:13:03 UTC - in response to Message 110287.  

I've been trying - and failing - to run these tasks, but I need cvmfs and it's not installing on Mint 21. I'll take a break, and then try installing it on the older and more stable Mint 20.3
ID: 110288 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110291 - Posted: 1 Nov 2022, 18:57:43 UTC

Mint 20.3 seems to take the cvmfs installation. Part 2 tomorrow.
ID: 110291 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2515
United Kingdom
Message 110292 - Posted: 1 Nov 2022, 19:03:09 UTC - in response to Message 110288.  

I've been trying - and failing - to run these tasks, but I need cvmfs and it's not installing on Mint 21. I'll take a break, and then try installing it on the older and more stable Mint 20.3
I have only been running Atlas using VB rather than natively. That took a long time till I tracked down the issue(s) causing problems.
ID: 110292 · Report as offensive
computezrmle

Send message
Joined: 2 Feb 22
Posts: 81
Germany
Message 110295 - Posted: 2 Nov 2022, 9:08:20 UTC - in response to Message 110288.  

I've been trying - and failing - to run these tasks, but I need cvmfs and it's not installing on Mint 21. ...

Did you allow user namespaces?
Required settings may vary a bit depending on the kernel version and the policy of your Linux vendor.


Did you install and enable the automounter service?


Did you install the recent cvmfs-config package beside the main package?
Use the format you prefer:
http://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default-latest.noarch.rpm
http://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default_latest_all.deb


Did you run (once) "cvmfs_config setup"?
Beside other steps it copies "cvmfs.auto" to the right location and configures the automounter to use it.
This needs to be checked.


Some systems now require a simple "sudo systemctl restart autofs" to work fine.
Others require a reboot.
Few systems require 2 reboots.


If a repository list is given in /etc/cvmfs/default.local the probes should now work.
e.g.
CVMFS_REPOSITORIES=cvmfs-config,atlas
Output of "cvmfs_config probe" should look like this:
Probing /cvmfs/cvmfs-config.cern.ch... OK
Probing /cvmfs/atlas.cern.ch... OK
ID: 110295 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110296 - Posted: 2 Nov 2022, 9:22:18 UTC - in response to Message 110295.  

Thanks for looking into this. I can post the error messages, if it would help? Where would you like them?

My second machine (Linux Mint 20.3, very similar hardware) accepted the cvmfs quickstart settings without complaint, and is currently an hour into its first ATLAS native task. It's recorded 4 hours 39 minutes of CPU time in that hour (6 cores, nothing else running, but slow to start the MT phase), so I think this one's OK.
ID: 110296 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110297 - Posted: 2 Nov 2022, 9:40:32 UTC

And having said that, it bailed out after a further 20 minutes. Uploaded and validated - task 368415294. Looks OK to me.
ID: 110297 · Report as offensive
computezrmle

Send message
Joined: 2 Feb 22
Posts: 81
Germany
Message 110298 - Posted: 2 Nov 2022, 10:36:27 UTC - in response to Message 110297.  

ATLAS tasks from LHC@home run through an initialization/configuration phase.
That phase is always singlecore even if BOINC (or vbox) treats it as n-core.

Depending on the "power", the total load and the network/internet load of the client computer it can take 3..25 min until ATLAS has set up everything that is required to start n (posix-)threads (called worker threads) that concurrently do the event processing.

Each task processes 200 events and each event can take between a few seconds and >20 min to be processed.
Once a worker has finished an event it immediately picks the next one from the event queue until that queue is empty.

At the end there's another singlecore phase that combines the results from all workers and reports a (small) logfile plus a (large) result file back to BOINC.


If you have trouble with CVMFS, it might be better to post questions and error messages at the LHC@home forum.


Your example task succeeded which can be seen close to the end of stderr.txt:
[2022-11-02 09:33:35] HITS file was successfully produced:

If this line is shown and the HITS file has a reasonable size (here: 160 MB) errors/warnings mentioned before can be ignored.
They are not BOINC related.
ID: 110298 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110299 - Posted: 2 Nov 2022, 10:54:40 UTC - in response to Message 110298.  

Thanks again. So, we can take it that cvmfs runs OK on Mint 20.3

But it doesn't on Mint 21. I get:

richard@Rosa:~$ sudo apt-get install cvmfs
[sudo] password for richard:         
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
cvmfs is already the newest version (2.9.4~1+ubuntu20.04).
The following packages were automatically installed and are no longer required:
  libflashrom1 libftdi1-2
Use 'sudo apt autoremove' to remove them.
0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade.
richard@Rosa:~$ sudo cvmfs_config setup
Error: failed to load cvmfs library, tried: './libcvmfs_fuse3_stub.so' '/usr/lib/libcvmfs_fuse3_stub.so' '/usr/lib64/libcvmfs_fuse3_stub.so' './libcvmfs_fuse_stub.so' '/usr/lib/libcvmfs_fuse_stub.so' '/usr/lib64/libcvmfs_fuse_stub.so'
./libcvmfs_fuse3_stub.so: cannot open shared object file: No such file or directory
libcrypto.so.1.1: cannot open shared object file: No such file or directory
/usr/lib64/libcvmfs_fuse3_stub.so: cannot open shared object file: No such file or directory
./libcvmfs_fuse_stub.so: cannot open shared object file: No such file or directory
libcrypto.so.1.1: cannot open shared object file: No such file or directory
/usr/lib64/libcvmfs_fuse_stub.so: cannot open shared object file: No such file or directory

Failed to read CernVM-FS configuration
Could you pass that back to the LHC team, please? All those fuse files seem to be present in /usr/lib:

ID: 110299 · Report as offensive
computezrmle

Send message
Joined: 2 Feb 22
Posts: 81
Germany
Message 110300 - Posted: 2 Nov 2022, 11:40:00 UTC - in response to Message 110299.  

Looks like Mint 21 installs newer libraries than those the ubuntu CVMFS package is compiled against.
I suspect nobody at CERN or LHC@home will solve those library issues and compile a package for a distribution not used there.

If a precompiled packet doesn't work the recommended way would be to get the CVMFS source from github and compile the client on the target computer.
See:
https://github.com/cvmfs/cvmfs

Since it's just for a test it may be too much work to do so, hence the test may be done only on Mint 20.3.
ID: 110300 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 110301 - Posted: 2 Nov 2022, 11:45:28 UTC - in response to Message 110300.  

Agreed. But since Mint is derived from Ubuntu, it might be worth keeping an eye on the next Ubuntu release, in case action is needed there,
ID: 110301 · Report as offensive
1 · 2 · Next

Message boards : Questions and problems : Effect of settings on multi-core tasks.

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.