Hello,
Curious if anyone has set up a BOINC upload server on a separate host to the rest of the BOINC server. How did you go about it etc?

Cheers
Seth

The answer depends on why you want to move the upload handler to a different server? The usual case is that you are limited by hard disk IO and want to split of IO intensive processes from each other.

If you have different applications in your project you might want to have a separate upload server for each application. Each server will need to have the FUH, validator, assimilator and file deleter for this application. The transition is a bit tricky but not so difficult after all.

If you have a high throughput storage system at your backend and instead the bottleneck seems to be webserver connections you can also migrate to a multi server setup.

Hi Christian,
Thanks for your response.

To be clear my backend is all in AWS so lots of levers to pull in terms of additional capacity.

My DB is running at about 2100 iops and I have provisioned for 3000iops. Database server humms away with plenty of spare CPU and Memory, its a m4.2XL

My WU gzipped files reside in S3 and are fetched by the clients directly from there using a URL.

Results files are pushed to S3 from my BOINC server every 5 minutes.

My remaining server is running on a x1.32xlarge, it seems to be coping well now with the load but going to hit a limit shortly I think. My WU take about 65 seconds to process on my AWS C size cleints. I was running at about 12000 WU per minute. I have 50 each of the transitioners and validators. I'm using a simple bitwise validator and a single instance is working well.

I'm not using FCGI, I have read some mixed thing on it which lead me to focus on some other optimization first.

I have read about moving transitioners/validators to other clients but unsure that will give me a big increase in terms of capacity.

The nature of my project means WU are small in size. It suits me anyway as I'm using an AWS spot fleet for my clients. The result of small files is lots of web server hits thus my interest in splitting the upload server component onto another host. I have an idea in my mind how to do this but if you have any gotchas or any other suggestions for performance tuning I would very much like to hear them.

Cheers
Seth

My remaining server is running on a x1.32xlarge, it seems to be coping well now with the load but going to hit a limit shortly I think. My WU take about 65 seconds to process on my AWS C size cleints. I was running at about 12000 WU per minute. I have 50 each of the transitioners and validators. I'm using a simple bitwise validator and a single instance is working well.

Do you mean that you have a single assimilator or a single validator?

I'm not using FCGI, I have read some mixed thing on it which lead me to focus on some other optimization first.

I have read about moving transitioners/validators to other clients but unsure that will give me a big increase in terms of capacity.

The nature of my project means WU are small in size. It suits me anyway as I'm using an AWS spot fleet for my clients. The result of small files is lots of web server hits thus my interest in splitting the upload server component onto another host. I have an idea in my mind how to do this but if you have any gotchas or any other suggestions for performance tuning I would very much like to hear them.

With this high throughput and short runtimes you should definitly think about FCGI. Just remember that for every connection of a client a new process has to spin up. With FCGI you would keep a number of processes running that would serve the requests without the need of starting new ones. This can be done with the stock CGI and file_upload_handler using multiwatch and spawn_fcgi.

Just keep in mind that the validator and assimilator need to access the data uploaded by the client so they should be on the same machine if the data is held locally. If you transition you need either make sure that all old WUs are processed before you change the upload URL in your template file or restrict the old validator, assimilator chain to only process WUs with an ID lower than what was the highest ID when you changed the upload URL (don't forget to stop the work generator).

Hi Christian,
Thanks for the tips.

I meant I have a single assimilator but I have seen an issue over the last day with assimilation that suggests to me the single instance was not coping with the load. I have work units marked as complete (success) in the boinc DB but the files are still in my upload directory. Also some results seem to have gone missing completely.

I have 50 validators running.

I'll move to FCGI tonight. I understand I need to upgrade my apache first correct?

To be clear are you saying that I don't need to rebuild any of my boinc components to start using fcgi? I've looked at some instructions here that say I need to rebuild some components: https://boinc.berkeley.edu/trac/wiki/MultiHost I'm a bit confused when you say it can be done with "stock CGI and file_upload_handler using multiwatch and spawn_fcgi"

Thanks again for your advice.

Cheers
Seth

Hi Christian,
I've followed the steps for FCGI but I keep seeing in my apache2 log the following:

FCGI: FCGI_Accept failed - exiting

Any ideas?

Seems to be triggered by download requests only (cgi?).

Cheers
Seth

BTW I'm using the instructions here, I think they are a little out of date: https://boinc.berkeley.edu/trac/wiki/MultiHost

Yep the instructions are out of date. It's running now.

I have asked before but don't think I got a reply. When I find mistakes or outdated instructions how can I contribute back to the project with updates?

Cheers
Seth

Easiest way is to ask David for an account on there to make changes yourself.
But else sign up to the BOINC projects email list and write your changes via there. Someone (Christian or me) will then probably pick it up and put it into place.

---

To be clear are you saying that I don't need to rebuild any of my boinc components to start using fcgi? I've looked at some instructions here that say I need to rebuild some components: https://boinc.berkeley.edu/trac/wiki/MultiHost I'm a bit confused when you say it can be done with "stock CGI and file_upload_handler using multiwatch and spawn_fcgi"

There are separate binaries for FCGI built if you have the fcgi development headers installed. It seems you already found them.