config.xml

Message boards : Server programs : config.xml
Message board moderation

To post messages, you must log in.

AuthorMessage
Seth

Send message
Joined: 19 Nov 16
Posts: 63
Australia
Message 77772 - Posted: 7 May 2017, 9:20:35 UTC

Hi Team,
Can anyone elaborate a little more what <feeder_query_size> is actually used for. Is there any relation to it and <shmem_work_items>

Cheers
Seth
ID: 77772 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15477
Netherlands
Message 77775 - Posted: 7 May 2017, 23:25:27 UTC - in response to Message 77772.  

See http://boinc.berkeley.edu/trac/wiki/ProjectOptions#Jobscheduling for the documentation, if you hadn't already.
ID: 77775 · Report as offensive
Seth

Send message
Joined: 19 Nov 16
Posts: 63
Australia
Message 77783 - Posted: 8 May 2017, 13:30:06 UTC - in response to Message 77775.  
Last modified: 8 May 2017, 13:31:38 UTC

Thanks Jord. I had looked at that but it doesn't really tell me anything.

For some reason it's implied that <feeder_query_size> should be double <shmem_work_items> but it doesn't explain why.

Is <feeder_query_size> actually used by the feeder? From what I understand the enumeration query is only really used by the transitioner?

Cheers
Seth
ID: 77783 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 321
Germany
Message 77785 - Posted: 8 May 2017, 16:26:48 UTC

I'm not sure why the query size is double the shmem size, I would need to look in the code for that. But the shmem_work_size is what it says the amount of tasks that are held in the shmem array that is shared between feeder and scheduler. The feeders job is it to fill this array with unsent tasks and therefore it regularly queries the database. This is what you limit with feeder_query_size, how much tasks the feeder retrieves from the DB to put into the shmem array. You want it to be bigger than item_size because the feeder needs to make sure that tasks for all enables applications are present in the array.
ID: 77785 · Report as offensive
Seth

Send message
Joined: 19 Nov 16
Posts: 63
Australia
Message 77834 - Posted: 10 May 2017, 10:31:02 UTC - in response to Message 77785.  
Last modified: 10 May 2017, 11:04:54 UTC

OK so have determined for a single project/app server the value of both these params just needs to be the same. A larger value for feeder_query_size is pointless.

Cheers
Seth
ID: 77834 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 321
Germany
Message 77837 - Posted: 10 May 2017, 12:36:11 UTC

It does not need to be the same. I wouldn't expect a larger feeder_query_size to be harmful. Even if feeder_query_size is lower than shmem item size it should still work but it would take some more time to fill an empty shmem segment. I'm not sure what you want to optimize for. I guess your feeder is not fast enough to refill the array as the schedulers are faster in sending out tasks. You could try to decrease the feeder sleeping time which seems to be a better value to optimize. I don't know if there is a commandline option for this or if you need to change this in the code.
ID: 77837 · Report as offensive
Seth

Send message
Joined: 19 Nov 16
Posts: 63
Australia
Message 77852 - Posted: 10 May 2017, 23:31:23 UTC - in response to Message 77837.  

HI Christian,
Thanks for your notes.

I had experimented before with sleep time, I think adding --sleep_interval N to the feeder command did the job.

With the large fleets of vCPU I'm running I exhaust a shared mem segment of 8000 every 5 seconds so I'm trying to push that limit while at the same time trying to manage my other bottle neck which is the MySQL server. There is no option to increase the hardware specs on the MySQL server, it's maxed out already, so looking at ways to reduce the load on it hence my questions on the size really needed for feeder_query_size.

Cheers
Seth
ID: 77852 · Report as offensive

Message boards : Server programs : config.xml

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.