Tasks max memory, temporarily freeze until swap invoked (linux)

Message boards : Questions and problems : Tasks max memory, temporarily freeze until swap invoked (linux)
Message board moderation

To post messages, you must log in.

AuthorMessage
Psnarf

Send message
Joined: 12 Nov 20
Posts: 2
Message 101552 - Posted: 12 Nov 2020, 19:06:07 UTC

I have a Ryzen 7 CPU with 8 cores, each of which can run 2 threads. Boinc probably queries /proc/cpuinfo and counts 16 cores, the 8 physical cores with 2 threads each, instead of 8 cores. That results in Boinc launching 16 simultaneous tasks, each with 2 threads possibly reaching 4Gb ram of the 16Gb physical ram. When that happens, the system starts swapping out processes using the swap partition. After a minute of freezing, I can get to a command prompt and stop the boinc processes. If I leave boincmgr open, it restarts all 32 threads and locks up the computer again.

Is there a way to limit boinc to use the 8 physical cores instead of launching 32 threads at once?
ID: 101552 · Report as offensive
Psnarf

Send message
Joined: 12 Nov 20
Posts: 2
Message 101553 - Posted: 12 Nov 2020, 19:15:05 UTC - in response to Message 101552.  

I tried "<max_ncpus_pct>50</max_ncpus_pct>" in global_prefs.xml, now I have 8 tasks, but each has 3 threads. Only one task uses 4Gb ram. I can live with that.
ID: 101553 · Report as offensive
Les Bayliss
Help desk expert

Send message
Joined: 25 Nov 05
Posts: 1654
Australia
Message 101554 - Posted: 12 Nov 2020, 19:59:44 UTC - in response to Message 101553.  

That's easy -

In Menu > Options > Computing preferences > Computing

The first item - Usage limits
Set a percentage that gives the number of cores that you want to use.
ID: 101554 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 101556 - Posted: 12 Nov 2020, 21:35:09 UTC - in response to Message 101552.  
Last modified: 12 Nov 2020, 21:36:00 UTC

I have a Ryzen 7 CPU with 8 cores, each of which can run 2 threads. Boinc probably queries /proc/cpuinfo and counts 16 cores, the 8 physical cores with 2 threads each, instead of 8 cores. That results in Boinc launching 16 simultaneous tasks, each with 2 threads possibly reaching 4Gb ram of the 16Gb physical ram.
..
If I leave boincmgr open, it restarts all 32 threads and locks up the computer again.
This I don't get. Unless you run a project which application double loads each CPU core, essentially BOINC detects 8 cores, 16 threads. It can therefore only allow a project to run 16 tasks at once, not 32. Which project(s) do you run?

Are you running an app_config.xml file for the affected project(s)? if so, what's in it/them?

I tried "<max_ncpus_pct>50</max_ncpus_pct>" in global_prefs.xml, now I have 8 tasks, but each has 3 threads.
Same thing here. You don't have to hand-edit global_prefs.xml, you can do that via the project's website->your account->Computing preferences. Or use the local preferences as Les points out. These override the web preferences.

But setting CPU used to 50% will make a 16 thread CPU an 8 thread CPU. It won't however triple load each thread by default, just single load, so 8 threads, 8 tasks. Not 24. Unless the project's science application stipulates to use more than one core/thread. But that's something the project's application does, not BOINC. Therefore again, which project(s) do you run?
ID: 101556 · Report as offensive
ProDigit

Send message
Joined: 8 Nov 19
Posts: 718
United States
Message 101645 - Posted: 19 Nov 2020, 16:14:32 UTC

For 16 threads it's better to have at least 8GB of RAM.
Some projects require a lot more, but most are using <500MB/thread.
ID: 101645 · Report as offensive
Profile Dave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2516
United Kingdom
Message 101648 - Posted: 19 Nov 2020, 16:41:19 UTC - in response to Message 101645.  

For 16 threads it's better to have at least 8GB of RAM.
Some projects require a lot more, but most are using <500MB/thread.


And some tasks Africa Rain Project with WCG and Hadam4, N216 tasks with CPDN really hammer level3 cache, the latter using about 4MB/task. This makes them really slow down if you use too many cores. With the CPDN ones, running using more threads than you have cores actually reduces throughput.
ID: 101648 · Report as offensive

Message boards : Questions and problems : Tasks max memory, temporarily freeze until swap invoked (linux)

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.