No way to run projects sequentially on host?

Message boards : Questions and problems : No way to run projects sequentially on host?
Message board moderation

To post messages, you must log in.

AuthorMessage
melk

Send message
Joined: 27 Jan 18
Posts: 1
Message 84565 - Posted: 27 Jan 2018, 18:16:21 UTC
Last modified: 27 Jan 2018, 18:16:37 UTC

Hi folks, so I have been looking into this issue for a while now, and it seems everything I read and everyone I talk to seems to indicate that if you run multiple projects on a host, there is NO way to run them sequentially? That is for example process Project A for 5 minutes, switch to project B for 5 minutes, switch to Project C, etc.

Currently on my CPU (4c/8t) it is processing 3 different WU/tasks from 3 different projects at the same time. This can't be good for efficiency inside the cpu? Wouldn't it be far better to process only 1 type of task/wu exclusively and then switch to the next?

Are there any plans to ever implement this type of task scheduling? Thanks!
ID: 84565 · Report as offensive     Reply Quote
Richard Haselgrove
Volunteer moderator
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 2764
United Kingdom
Message 84567 - Posted: 27 Jan 2018, 18:43:42 UTC - in response to Message 84565.  

On the contrary, I feel that greater efficiency can sometimes be achieved by running different projects together.

Some programs rely primarily on integer mathematics, others require floating point maths.

Some programs are heavily optimised, making extensive use of SIMD (single instruction, multiple data) operations. Others, not so much.

Some projects (exploring the effect of parameters on an algorthm) require very little data: others have to load large datasets.


It's perhaps easiest to consider some hypothetical project which makes extremely heavy use of a single resource within the computing environment - it might be memory access, it might be the CPU's floating point unit. If multiple copies of that single project are running together, the stress will be multiplied. The memory bus may become saturated, the CPU may need thermal or power throttling.

On the other hand, if that 'heavy' project is run alongside a 'lighter' project, say an integer maths algorithmic task, both may be able to run at full speed in their respective parts of the hardware.
ID: 84567 · Report as offensive     Reply Quote
whynot

Send message
Joined: 8 May 10
Posts: 82
Ukraine
Message 84646 - Posted: 3 Feb 2018, 18:34:06 UTC - in response to Message 84567.  


On the contrary, I feel that greater efficiency can sometimes be achieved by running different projects together.


Pity we can't experiment with differnt schedulers, because we have one scheduler that fits everyone (guess what, it doesn't). Once, after WU supply problems (because network problems outside of my reach and unrelated project management difficulties, outside my reach too) the scheduler went into some kind of trottling mode and started running one project at a time (8 cores). To my surprise app from one paticular project went down from ~3.2h/WU to ~2.9h/WU.

After some thinking I've got to conclusion that there is one resource that all code/data is sharing. It is L2 cache.

What is good thing, I've just gave up on micro-managing (more like micro-massaging) the scheduler.
I'm counting for science,
points just make me sick.
ID: 84646 · Report as offensive     Reply Quote
Profile Gary Charpentier
Avatar

Send message
Joined: 23 Feb 08
Posts: 1569
United States
Message 84661 - Posted: 4 Feb 2018, 17:10:11 UTC - in response to Message 84646.  
Last modified: 4 Feb 2018, 17:13:10 UTC


On the contrary, I feel that greater efficiency can sometimes be achieved by running different projects together.


Pity we can't experiment with differnt schedulers, because we have one scheduler that fits everyone (guess what, it doesn't). Once, after WU supply problems (because network problems outside of my reach and unrelated project management difficulties, outside my reach too) the scheduler went into some kind of trottling mode and started running one project at a time (8 cores). To my surprise app from one paticular project went down from ~3.2h/WU to ~2.9h/WU.

After some thinking I've got to conclusion that there is one resource that all code/data is sharing. It is L2 cache.

What is good thing, I've just gave up on micro-managing (more like micro-massaging) the scheduler.

What human thinks of as an ideal schedule rarely is. https://en.wikipedia.org/wiki/Scheduling_(computing) The problem has had a lot of thought over time. Obviously you aren't too upset with the O/S's scheduler, but you don't see inside it either.

Without putting the code into something like valgrind and actually studying what is going on, perhaps best to simply leave it alone as there are an infinite number of way to make it slower and perhaps only zero ways to make it better.

BTW, Boinc is open source so if you with to put a different scheduler in, go right ahead.

As to sequential running, set it to run on one core, change the switch between tasks time to something bigger than the longest task a project sends. Your RAC will be low, but you have your answer. If you want you could run an instance of Boinc for each core with those settings and increase your RAC that way. You might even set each instance so it only gets work from one project. I believe there are enough switches to allow this, but I'll leave researching how to do this up to you.
ID: 84661 · Report as offensive     Reply Quote
whynot

Send message
Joined: 8 May 10
Posts: 82
Ukraine
Message 84737 - Posted: 10 Feb 2018, 15:35:46 UTC - in response to Message 84661.  


What human thinks of as an ideal schedule rarely is. https://en.wikipedia.org/wiki/Scheduling_(computing) The problem has had a lot of thought over time. Obviously you aren't too upset with the O/S's scheduler, but you don't see inside it either.


Yes, I can: /usr/share/doc/linux-doc-*/Documentation/scheduler/


BTW, Boinc is open source so if you with to put a different scheduler in, go right ahead.


Well, that clearly describes difference between Free and Open. Anyway, I'm playing infamous Must-Do-Something-For-Living card here.
I'm counting for science,
points just make me sick.
ID: 84737 · Report as offensive     Reply Quote

Message boards : Questions and problems : No way to run projects sequentially on host?

Copyright © 2018 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.