BOINC 7.0.40-42 and new app_config.xml

Message boards : BOINC client : BOINC 7.0.40-42 and new app_config.xml
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 46666 - Posted: 8 Dec 2012, 21:08:11 UTC
Last modified: 8 Dec 2012, 21:24:09 UTC

Just downloaded the new version to specifically test the app_info.xml and see if I could get the system to not allow more of 1 CEP2 [one of those IO intense]. Here's the sample content:

<app_config>
   <app>
      <name>cep2</name>
      <max_concurrent>1</max_concurrent>
      <gpu_versions>
          <gpu_usage>.25</gpu_usage>
          <cpu_usage>.1</cpu_usage>
      </gpu_version>
   </app>
</app_config>


The (short) app name I pulled from the client_state.xml to be sure and placed a copy of the file in both the main data_dir and, following the app_info.xml logic, also in the WCG project folder [no mention in wiki where to put]. When I suspended a running task to see if the next in line cep2 would not start, it did. Don't know if the setup is incorrect, or if the feature is not yet operational. No errors of any kind were logged.

edit: Title to enter proper version number.
ID: 46666 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15476
Netherlands
Message 46670 - Posted: 8 Dec 2012, 23:16:01 UTC - in response to Message 46666.  

Not sure, but would this not require:
1. a restart of the client?
2. new tasks to be downloaded and used by it?
ID: 46670 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 46671 - Posted: 8 Dec 2012, 23:40:32 UTC - in response to Message 46670.  

The client was restarted with app_config.xml in the project folder before the first CEP2 job was pulled. When the 2nd CEP2 was pulled and manually started [against expectation it did] copied the config file to the main data_dir and restarted client again. 2 CEP2 kept running.

No matter, the object is to stop more than N to run concurrent, not to control how many are fetched. Can't see the relevance of how many there are in the buffer, the whole purpose WCG asked for this feature to be added, so volunteers don't have to cumbersomely control how many heavy duty to set the device profile to as maximum to send. Anyway, it's not working for me, and looking for any reports that show it does, or hints as to what may be wrong with the file format/placement, or also possible, it's not yet enabled, of course GFB, there's a bug.

My expectation was that the 2nd CEP2 job would be skipped, to start the next after in line [per FIFO]. It did not.
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 46671 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 46681 - Posted: 9 Dec 2012, 10:50:24 UTC - in response to Message 46671.  
Last modified: 9 Dec 2012, 11:13:17 UTC

A user at WCG pointed out that the wiki from which I copied the template has a closing tag error, a missing s in </gpu_versions>. He's got it to work as expected, to the point that if max per setting is running and no other is buffered, threads will go idle. In his example had 2 SIMAP running, 4 CEP2 buffered, and app_config set to allow only 1 CEP2 to run at a time. His quad had 1 idle core.

edit: OK, to confirm, it was the missing s in the closing tag. Current setting of max 1 and no more will start.
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 46681 · Report as offensive
MarkJ
Volunteer tester
Help desk expert

Send message
Joined: 5 Mar 08
Posts: 272
Australia
Message 46682 - Posted: 9 Dec 2012, 11:28:16 UTC

Just a quick question on this. In the doco it mentions the tags needed, are they all required? Are the <gpu_versions> tags required?
MarkJ
ID: 46682 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 46683 - Posted: 9 Dec 2012, 11:35:30 UTC - in response to Message 46681.  

A user at WCG pointed out that the wiki from which I copied the template has a closing tag error, a missing s in </gpu_versions>. He's got it to work as expected, to the point that if max per setting is running and no other is buffered, threads will go idle. In his example had 2 SIMAP running, 4 CEP2 buffered, and app_config set to allow only 1 CEP2 to run at a time. His quad had 1 idle core.

edit: OK, to confirm, it was the missing s in the closing tag. Current setting of max 1 and no more will start.

Wiki updated - thanks.
ID: 46683 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15476
Netherlands
Message 46684 - Posted: 9 Dec 2012, 12:57:04 UTC

Well, on documentation, apparently we're not to use that one, but the 'official' one at http://boinc.berkeley.edu/wiki/Client_configuration#Application_configuration .. yeah, I don't get it either. ;-)
ID: 46684 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 46688 - Posted: 9 Dec 2012, 14:11:45 UTC - in response to Message 46684.  
Last modified: 9 Dec 2012, 14:18:41 UTC

Well, on documentation, apparently we're not to use that one, but the 'official' one at http://boinc.berkeley.edu/wiki/Client_configuration#Application_configuration .. yeah, I don't get it either. ;-)

That's the one which starts "There are three configuration mechanisms...", numbered

1 Configuration file
1.1 Logging flags
1.2 Options
2 Command-line options
3 Environment variables
4 Application configuration

? ;-)

Edit - the document we've all been editing is directly linked from the front page of Software Development (http://boinc.berkeley.edu/trac/wiki/SoftwareDevelopment?version=127#Currentdevelopment), so I think it's as well that we keep it accurate, too.
ID: 46688 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15476
Netherlands
Message 46694 - Posted: 9 Dec 2012, 19:05:18 UTC - in response to Message 46688.  

Edit - the document we've all been editing is directly linked from the front page of Software Development (http://boinc.berkeley.edu/trac/wiki/SoftwareDevelopment?version=127#Currentdevelopment), so I think it's as well that we keep it accurate, too.

Yeah, but for that David told Rom, Charlie and me that documentation for the client only needs to go into the Wiki only. Server software/project related stuff goes into the Trac/Wiki, so he's been setting this thing up in the wrong place.

I already pointed that out, don't be surprised if the one in Trac goes missing all of a sudden. ;-)
ID: 46694 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 46716 - Posted: 11 Dec 2012, 17:32:43 UTC
Last modified: 11 Dec 2012, 17:33:31 UTC

I've been playing around with this feature, and it seems to be very useful in two quite different forms.

1) For CPU-only apps.

<app_config>
<app>name</app>
<max_instances>N</max_instances>
</app>
</app_config>

As many users will have noticed, when a project is underworked in BOINC v7 (either a newly attached project, or following a server outage), the client has a tendency to both fetch and schedule that project exclusively until REC and priority start to balance out with other pre-existing projects. This usage of app_config can help to even out the balance between projects, though I fear boinc may starve low-priority projects while the re-balancing process finishes - I'm testing this.

2) For GPU-only apps.

<app_config>
<app>name</app>
<gpu_versions>
<gpu_usage>.xx</gpu_usage>
<cpu_usage>.y</cpu_usage>
</gpu_versions>
</app>
</app_config>

has the effect of supplying a <count> value, and allowing multiple copies of a GPU app to run on a single GPU resource - something popular with GPU users. This is much easier and safer than using a full app_info.xml file, because it doesn't disable the automatic downloading of a new application version if the project server issues one.
ID: 46716 · Report as offensive
Jim1348

Send message
Joined: 8 Nov 10
Posts: 310
United States
Message 46717 - Posted: 11 Dec 2012, 19:07:36 UTC - in response to Message 46716.  


2) For GPU-only apps.

<app_config>
<app>name</app>
<gpu_versions>
<gpu_usage>.xx</gpu_usage>
<cpu_usage>.y</cpu_usage>
</gpu_versions>
</app>
</app_config>



Thanks, that is what I have been waiting for. But I had to use a version closer to what SekeRob used, to allow two HCC work units to run on my GPU:

<app_config>
  <app>  
  <name>hcc1</name>
  <user_friendly_name>Help Conquer Cancer</user_friendly_name>
	<gpu_versions>
	<gpu_usage>0.5</gpu_usage>
	<cpu_usage>1.0</cpu_usage>
	</gpu_versions>
   </app>
</app_config>


(And I had to place it in both the BOINC data folder, and the project folder.)
ID: 46717 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 46718 - Posted: 11 Dec 2012, 19:30:31 UTC

Ooops, my bad. I had looked back at an earlier version of the documentation, and forgot to come back forward to the present day before copying and posting.

Try these instead - and let's hope I don't get duplicated </app> tags this time.

1) For CPU-only apps.

<app_config>
<app>
<name>name</name>
<max_concurrent>N</max_concurrent>
</app>
</app_config>


2) For GPU-only apps

<app_config>
<app>
<name>name</name>
<gpu_versions>
<gpu_usage>.xx</gpu_usage>
<cpu_usage>.y</cpu_usage>
</gpu_versions>
</app>
</app_config>


From the changesets, it looks as if you may have to use BOINC v7.0.41 (or v7.0.42, just available) for the GPU version to work consistently.
ID: 46718 · Report as offensive
Crystal Pellet

Send message
Joined: 23 Apr 12
Posts: 41
Netherlands
Message 46721 - Posted: 12 Dec 2012, 7:39:24 UTC - in response to Message 46718.  
Last modified: 12 Dec 2012, 7:43:06 UTC

Hi Richard,

It seems you've the lead with this new feature testing ;)

I discovered a new unexpected feature (7.0.41).
A task that may not start because of the max_concurrent setting, is still counted for the work buffer for other projects.
With a low cache that will lead to idle cores.

Running on a quad 100% CPU/all processors and a low cache of 0.01.
No GPU at the moment.
After the last GPU task ended only 3 CPU tasks running and 1 WCG CEP2 waiting (max 1 and already 1 running).
Because of the estimated time for that waiting task is 11 hours and my low cache, it will not ask work for other projects, so 1 idle core as long as that task isn't started.

CP
ID: 46721 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 46724 - Posted: 12 Dec 2012, 9:37:26 UTC - in response to Message 46721.  

Already we've made that conclusion off-line in discussion with knreed **. Also, if the user goes wild, too much work could get cached of the max_concurrent and eventually go "too late". As the developer indicated, something would give. How's the work submitting project to know the user has set concurrency restrictions... the buffer is the buffer, and if full it's full. Backup project, or at least another project outside the most wanted, to fill those instances.

In the case of CEP2, where we have the fortune of being able to control how many "in progress", I've set the number to 1.5 times a single thread on a 1 day buffer. So far, it's running 24/7 seamless CEP2 on one thread [now on 7.0.42]. Sometimes 2 are ready to start [because a CEP2 job ran shorter than normal], sometimes 3. The user is in charge to tweak the settings. So far, where I've set other sciences to run 4 max concurrent on an 8 core, not walked into the idle yet, but that's with a 5 way WCG science mix... total 43 tasks waiting in queue. Works wonderful so far.

** Some stuff I've been made to understand will have to wait for 7.2. The focus is now to get a client that does not require the app_info hoopla for multiple GPU tasks and get rid of the anonymous platform causing problems. The second focus is to limit the IO intense, speak CEP2 to not run on more threads than specified, to prevent inefficiency and or user impairment when crunching during use is allowed. Think both issues have been met.
ID: 46724 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5077
United Kingdom
Message 46728 - Posted: 12 Dec 2012, 11:49:43 UTC

I haven't noticed any problems with CPU apps, but for GPU apps there seems to be perhaps a '<' test where there should be a '<=' test.

<gpu_usage>0.50</gpu_usage>

only allowed one task to run, but

<gpu_usage>0.49</gpu_usage>

allowed two tasks.
ID: 46728 · Report as offensive
Crystal Pellet

Send message
Joined: 23 Apr 12
Posts: 41
Netherlands
Message 46729 - Posted: 12 Dec 2012, 12:26:47 UTC - in response to Message 46724.  
Last modified: 12 Dec 2012, 12:29:52 UTC

Backup project, or at least another project outside the most wanted, to fill those instances.

Rob, if you don't mind I'm not a sole WCG-cruncher, so my backup project in my example was SIMAP.
All my projects have an equal resource share.
In my example I expected to get another SIMAP task fetched, but as you said "a filled buffer" is a full buffer, even if a task in the buffer isn't allowed to start.
It's just the user should be aware of this and not use too low buffer sizes when using the max_concurrent or Rom/David could work around in case of idle core(s), that may be used.

@Richard: For me 0.5 resulted in 2 running GPU-tasks.

Will load 7.0.42 as soon as the next checkpoint is written for a CEP2-task. Waiting for hours and hours already :(

Suggestion: Why not reread all app_config.xml's when "read config file".
That way you don't have to restart BOINC.
ID: 46729 · Report as offensive
Crystal Pellet

Send message
Joined: 23 Apr 12
Posts: 41
Netherlands
Message 46732 - Posted: 12 Dec 2012, 13:29:45 UTC

app_config.xml is first ignored after a reset of the project.
The mentioned apps should at least once be stored in the client_state.

A bit confusing text in message event "World Community Grid 12 dec 14:19:42 app cep2 not found in app_config.xml"

The app cep2 is in the app_config.xml, but not in the client_state.xml
ID: 46732 · Report as offensive
SekeRob2

Send message
Joined: 6 Jul 10
Posts: 585
Italy
Message 46734 - Posted: 12 Dec 2012, 14:17:53 UTC - in response to Message 46732.  
Last modified: 12 Dec 2012, 14:56:15 UTC

Crystal Pellet, I'd thought the idle core would initiate a call to a backup project, but as you surmised too, there being no buffer room. If that weakness could be overridden for a backup project task [those that only supply minimal work to keep a core occupied], that would be the cream on top of the icing on the cake.:D

edit: Something the developers may want to think about [if only for a blond moment]: Presently a client restart is needed to read in the app_config.xml [mistakenly, thought that the RPC call, rereading the settings, came from disk and not from memory cache]. My thinking is that if doing the "read config file" on the advanced menu, could that menu option be expanded to also read in the app_config files, for instant application?
Coelum Non Animum Mutant, Qui Trans Mare Currunt
ID: 46734 · Report as offensive
nanoprobe

Send message
Joined: 9 Apr 12
Posts: 51
United States
Message 46747 - Posted: 12 Dec 2012, 23:23:00 UTC
Last modified: 12 Dec 2012, 23:28:24 UTC

NM! I had a major brain cramp.
ID: 46747 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15476
Netherlands
Message 46755 - Posted: 13 Dec 2012, 6:22:35 UTC

People, you can stop PMing me with requests that I forward all your enhancement request information to the developers. I already did that mostly based on the contents of this thread.
ID: 46755 · Report as offensive
1 · 2 · 3 · Next

Message boards : BOINC client : BOINC 7.0.40-42 and new app_config.xml

Copyright © 2024 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.