Posts by Seth

21) Message boards : Server programs : config.xml (Message 77834)
Posted 10 May 2017 by Seth
Post:
OK so have determined for a single project/app server the value of both these params just needs to be the same. A larger value for feeder_query_size is pointless.

Cheers
Seth
22) Message boards : Server programs : config.xml (Message 77783)
Posted 8 May 2017 by Seth
Post:
Thanks Jord. I had looked at that but it doesn't really tell me anything.

For some reason it's implied that <feeder_query_size> should be double <shmem_work_items> but it doesn't explain why.

Is <feeder_query_size> actually used by the feeder? From what I understand the enumeration query is only really used by the transitioner?

Cheers
Seth
23) Message boards : Server programs : config.xml (Message 77772)
Posted 7 May 2017 by Seth
Post:
Hi Team,
Can anyone elaborate a little more what <feeder_query_size> is actually used for. Is there any relation to it and <shmem_work_items>

Cheers
Seth
24) Message boards : Server programs : db_purge and remote files (Message 77361)
Posted 13 Apr 2017 by Seth
Post:
Hi Guys,
I'd like to tidy up my DB size via db_purge. Reading the documentation for this I see:

"Workunits are purged only when their input files have been deleted"

How does this work when you have chosen to host input files on a remote HTTP server?

All I really want to do is remove completed results/workunits from my DB that were completed more than 7 days ago.

Cheers
Seth
[/b]
25) Message boards : Server programs : Duplicate entry '67596-38' for key 'hap' (Message 75862)
Posted 11 Feb 2017 by Seth
Post:
I'll see if it can catch it again and get a bigger log.

To be clear I'm prebuilding an AMI with BOINC installed and using it for each of my ec2 VM I spin up. To ensure some uniqueness around host id I remove client_state.xml from the prebuilt image so as to trigger generation of a unique host key again. It seems to work but I wonder if this might be triggering the problem too.

Cheers
Seth
26) Message boards : Server programs : and and any other performance tuning tips please (Message 75861)
Posted 11 Feb 2017 by Seth
Post:
Hi Christian,
Just finished a great performance run where 2,000 seemed to be the sweet spot for me. It was choking at 1,000 mainly as new clients were spinning up as they tend to download more work at their startup.

I had around 23K vCPU running at those settings and begun to be throttled by my DB (need a bigger one!).

It's good!

Cheers
Seth
27) Message boards : Server programs : and and any other performance tuning tips please (Message 75835)
Posted 11 Feb 2017 by Seth
Post:
Hi Team,
Any tips for optimizing these values. I'm thinking the defaults of 100 and 200 are probably to low for my server but imagine there might be a sweet spot maximum value before scheduler performance starts degrading.

To put this into context I'm trying to test with tens of thousands of clients with WU that take 70 seconds to process and the scheduler is struggling often returning zero work units to clients at high loads.

I've already tune other aspects of my server around number of validators/assimilators/deleters, fastcgi, winding down logging, hosting files on a different server (S3 with CloudFront on AWS), plenty of iops on my DB and large DB host.

Any performance tips appreciated.

Cheers
Seth
28) Message boards : Server programs : Duplicate entry '67596-38' for key 'hap' (Message 75823)
Posted 10 Feb 2017 by Seth
Post:
Anyone seen the error: Duplicate entry '67596-38' for key 'hap'

Cheers
Seth
29) Message boards : Server programs : Custom Deadlines (Message 75721)
Posted 5 Feb 2017 by Seth
Post:
Hi Christian,
Sorry I should have been more specific. I meant how can I control the deadline when I create the work unit. I'm using the create_work binary supplied with BOINC so just guess I add --delay_bound argument with a value in seconds?

Cheers
Seth
30) Message boards : Server programs : Custom Deadlines (Message 75707)
Posted 4 Feb 2017 by Seth
Post:
Hi,
Is there anyway to set the deadline for a WU at load time. The default is 10 days but what if I wanted to set this to 2 (or some other value)? Do I have to manually do this in the DB?

Cheers
Seth
31) Message boards : Server programs : BOINC Upload Server on Seperate Host (Message 75613)
Posted 2 Feb 2017 by Seth
Post:
Yep the instructions are out of date. It's running now.

I have asked before but don't think I got a reply. When I find mistakes or outdated instructions how can I contribute back to the project with updates?

Cheers
Seth
32) Message boards : Server programs : BOINC Upload Server on Seperate Host (Message 75611)
Posted 2 Feb 2017 by Seth
Post:
BTW I'm using the instructions here, I think they are a little out of date: https://boinc.berkeley.edu/trac/wiki/MultiHost
33) Message boards : Server programs : BOINC Upload Server on Seperate Host (Message 75610)
Posted 2 Feb 2017 by Seth
Post:
Hi Christian,
I've followed the steps for FCGI but I keep seeing in my apache2 log the following:

FCGI: FCGI_Accept failed - exiting

Any ideas?

Seems to be triggered by download requests only (cgi?).

Cheers
Seth
34) Message boards : Server programs : BOINC Upload Server on Seperate Host (Message 75572)
Posted 31 Jan 2017 by Seth
Post:
Hi Christian,
Thanks for the tips.

I meant I have a single assimilator but I have seen an issue over the last day with assimilation that suggests to me the single instance was not coping with the load. I have work units marked as complete (success) in the boinc DB but the files are still in my upload directory. Also some results seem to have gone missing completely.

I have 50 validators running.

I'll move to FCGI tonight. I understand I need to upgrade my apache first correct?

To be clear are you saying that I don't need to rebuild any of my boinc components to start using fcgi? I've looked at some instructions here that say I need to rebuild some components: https://boinc.berkeley.edu/trac/wiki/MultiHost I'm a bit confused when you say it can be done with "stock CGI and file_upload_handler using multiwatch and spawn_fcgi"

Thanks again for your advice.

Cheers
Seth
35) Message boards : Server programs : BOINC Upload Server on Seperate Host (Message 75539)
Posted 28 Jan 2017 by Seth
Post:
Hi Christian,
Thanks for your response.

To be clear my backend is all in AWS so lots of levers to pull in terms of additional capacity.

My DB is running at about 2100 iops and I have provisioned for 3000iops. Database server humms away with plenty of spare CPU and Memory, its a m4.2XL

My WU gzipped files reside in S3 and are fetched by the clients directly from there using a URL.

Results files are pushed to S3 from my BOINC server every 5 minutes.

My remaining server is running on a x1.32xlarge, it seems to be coping well now with the load but going to hit a limit shortly I think. My WU take about 65 seconds to process on my AWS C size cleints. I was running at about 12000 WU per minute. I have 50 each of the transitioners and validators. I'm using a simple bitwise validator and a single instance is working well.

I'm not using FCGI, I have read some mixed thing on it which lead me to focus on some other optimization first.

I have read about moving transitioners/validators to other clients but unsure that will give me a big increase in terms of capacity.

The nature of my project means WU are small in size. It suits me anyway as I'm using an AWS spot fleet for my clients. The result of small files is lots of web server hits thus my interest in splitting the upload server component onto another host. I have an idea in my mind how to do this but if you have any gotchas or any other suggestions for performance tuning I would very much like to hear them.

Cheers
Seth
36) Message boards : Server programs : BOINC Upload Server on Seperate Host (Message 75501)
Posted 26 Jan 2017 by Seth
Post:
Hello,
Curious if anyone has set up a BOINC upload server on a separate host to the rest of the BOINC server. How did you go about it etc?

Cheers
Seth
37) Message boards : Server programs : Apache MPM settings (Message 75407)
Posted 20 Jan 2017 by Seth
Post:
Just wondering which Apache MPM settingings you use and why?

Cheers
Seth
38) Message boards : Server programs : New Client not downloading much work at all (Message 75405)
Posted 20 Jan 2017 by Seth
Post:
Hi Rebirther,
No crashes for me at 2000.

Cheers
Seth
39) Message boards : Server programs : New Client not downloading much work at all (Message 75315)
Posted 14 Jan 2017 by Seth
Post:
Thanks Jord!

I seem to have overcome the limitations I was experiencing but it's unclear exactly what change I made to the config had an impact. Certainly changing:

<shmem_work_items>1000</shmem_work_items>
<feeder_query_size>2000</feeder_query_size>

Has made a difference plus there was an error in my cc_config.xml that I fixed.

My AWS C4.8xlarge BOINC client instance that was exhibiting the problems previously has been running fine now for over 2 hours so fingers crossed there.

Cheers
Seth
40) Message boards : Server programs : New Client not downloading much work at all (Message 75313)
Posted 14 Jan 2017 by Seth
Post:
There's nothing wrong with the client requests.


Previous 20 · Next 20

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.