and and any other performance tuning tips please

Message boards : Server programs : and and any other performance tuning tips please
Message board moderation

To post messages, you must log in.

AuthorMessage
Seth

Send message
Joined: 19 Nov 16
Posts: 44
Australia
Message 75835 - Posted: 11 Feb 2017, 0:46:54 UTC

Hi Team,
Any tips for optimizing these values. I'm thinking the defaults of 100 and 200 are probably to low for my server but imagine there might be a sweet spot maximum value before scheduler performance starts degrading.

To put this into context I'm trying to test with tens of thousands of clients with WU that take 70 seconds to process and the scheduler is struggling often returning zero work units to clients at high loads.

I've already tune other aspects of my server around number of validators/assimilators/deleters, fastcgi, winding down logging, hosting files on a different server (S3 with CloudFront on AWS), plenty of iops on my DB and large DB host.

Any performance tips appreciated.

Cheers
Seth
ID: 75835 · Report as offensive
ChristianB
Volunteer moderator
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 305
Germany
Message 75857 - Posted: 11 Feb 2017, 14:22:05 UTC

Hi Seth,
can you elaborate what values you want to tweak? My guess is you want to extend the shared memory array that keeps the work in RAM because your feeder is not fast enough for the scheduler. That is usually not a problem and only limited by the amount of RAM your server has. I've seen setups with 500 or even 800 slots. That should help in your case where there is a high throughput in the system. Just increase it and find the right spot for your project.
ID: 75857 · Report as offensive
Seth

Send message
Joined: 19 Nov 16
Posts: 44
Australia
Message 75861 - Posted: 11 Feb 2017, 14:31:20 UTC - in response to Message 75857.  

Hi Christian,
Just finished a great performance run where 2,000 seemed to be the sweet spot for me. It was choking at 1,000 mainly as new clients were spinning up as they tend to download more work at their startup.

I had around 23K vCPU running at those settings and begun to be throttled by my DB (need a bigger one!).

It's good!

Cheers
Seth
ID: 75861 · Report as offensive

Message boards : Server programs : and and any other performance tuning tips please

Copyright © 2017 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.