Cloning a BOINC preinstalled VM

Message boards : Questions and problems : Cloning a BOINC preinstalled VM
Message board moderation

To post messages, you must log in.

AuthorMessage
FritzBox360

Send message
Joined: 20 Apr 17
Posts: 4
Germany
Message 77500 - Posted: 20 Apr 2017, 9:01:56 UTC

Hello,

we would like to support our ressources to BOINC. Our computer labs have a virtualisation infrastructure via VMware over OpenSLX. As a result of that we can create a single VM ( e.g. Linux / Ubuntu ) and install BOINC there. After that we upload this VM to our servers and every client will get these VM.

We run into the problem, that all of our testing PCs had the same ID and were working at the same tasks. After a while the tasks failed.

Inn our opinion it is sth. with the computer ID or another ID which should be unique for the clients/servers

We found this --> https://boinc.berkeley.edu/wiki/Host_identification_and_merging

And we are thinking about to configure the .xml files. We would like each client to connect to one account, assign a project and start working individual tasks.

How is this possible? It would be fine if we had one preconfiggured VM that we just need to start on each client.

If you have any questions about this - just feel free to ask!

Thanks a lot
ID: 77500 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15478
Netherlands
Message 77502 - Posted: 20 Apr 2017, 12:56:09 UTC - in response to Message 77500.  

When you install BOINC on a system, and have it contact a project it gets a hostID and CPID appointed that's based on the hardware, the operating system and the MAC address of the network.

When you make a VM with the OS and BOINC installed and contacted a project, then start cloning that, all BOINC have the same hostID and CPID.

So the solution would be to delete all *.xml files in the clones and manually install BOINC in each/add the projects to it.
I'll see if I can get my colleague Christian to come by to explain in more detail.
ID: 77502 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 321
Germany
Message 77503 - Posted: 20 Apr 2017, 13:42:58 UTC

This is not easily explained as there are multiple posibilities depending on your specific setup. I would suggest to schedule a phone or skype call to talk about this in person (possibly in German too).
ID: 77503 · Report as offensive
FritzBox360

Send message
Joined: 20 Apr 17
Posts: 4
Germany
Message 77504 - Posted: 20 Apr 2017, 13:52:36 UTC

Hi guys,

thank you for the feedback.

Deleting all the .xml files would be no problem. But how can i automaticaly add a project? We have about 120 possible clients and it would be to hard to go to each client.

We have checked the boincmd but could not get it working.

Each client ( VM ) has an individual MAC - so it should be possible to create new IDs but we could not figure a way for it.

A skype call in german sounds nice :) - let me know if this is possible!

Thanks a lot!
ID: 77504 · Report as offensive
HAL9000
Help desk expert
Avatar

Send message
Joined: 13 Jun 14
Posts: 81
United States
Message 77511 - Posted: 21 Apr 2017, 3:08:58 UTC
Last modified: 21 Apr 2017, 3:11:17 UTC

What you are trying to do sounds much like what I have done on my VM server.

My process:
1. Create OS environment. OS was configured use the hostname of BNC-xxxxxx. Where xxxxxx is the last 6 of the MAC address from the VM.
2. Add BOINC
3. Configure BOINC client. Run benchmarks, set permissions, and configure for remote access.
4. Create start script for BOINC that attaches BOINC client to desired projects after BOINC starts.
NOTE: BOINC should not be attached to any projects until after the image is saved.
5. Save OS image.
6. Launch multiple instances of VM image.
7. Crunch lots of projects data.

sample BOINC start script (for Windows VMs)
boinc --detach
timeout 30
boinccmd --project_attach http://project1 auth
boinccmd --project_attach http://project2 auth
boinccmd --project_attach http://project3 auth
ID: 77511 · Report as offensive
FritzBox360

Send message
Joined: 20 Apr 17
Posts: 4
Germany
Message 77515 - Posted: 21 Apr 2017, 7:54:15 UTC - in response to Message 77511.  

thank you - we tried this!

Still the same problem that we have the same cpid and the clients are running the same tasks.

Is there no "easy" way to create just new IDs e.g. at startup or with a specific command?

We are running out of ideas here. Just trying to spend some cpu power but if all clients doing the same task this would be more than wasteful ...

Any ideas here?
ID: 77515 · Report as offensive
ChristianB
Volunteer developer
Volunteer tester

Send message
Joined: 4 Jul 12
Posts: 321
Germany
Message 77516 - Posted: 21 Apr 2017, 8:01:31 UTC

Automatically adding projects can be done again in multiple ways. If you don't want to use a static list you could use an account manager or if you have a static list you can provide an initialization file.

HAL9000: with this process you should also run into the one host-CPID for all cloned hosts issue as the host-CPID is generated at first startup and is based on the first available MAC. If you clone the VM after this was set all clones will use the same host-CPID. The Client can generate a new host-CPID if it detects a clash but that will only work if the first available MAC on the cloned VM changed after cloning from the original VM.

FritzBox360: I'll send you a PM with my skype username.
ID: 77516 · Report as offensive
Profile Jord
Volunteer tester
Help desk expert
Avatar

Send message
Joined: 29 Aug 05
Posts: 15478
Netherlands
Message 77521 - Posted: 21 Apr 2017, 10:26:31 UTC - in response to Message 77516.  

Thanks for picking up on this Christian. Afterwards, we should probably look and see if we need some documentation on this in the BOINC User Manual, or in the BOINC Wiki. I don't mind writing it but then I'll need to know what it's about. :-)
ID: 77521 · Report as offensive
HAL9000
Help desk expert
Avatar

Send message
Joined: 13 Jun 14
Posts: 81
United States
Message 77536 - Posted: 22 Apr 2017, 20:09:57 UTC - in response to Message 77516.  

Automatically adding projects can be done again in multiple ways. If you don't want to use a static list you could use an account manager or if you have a static list you can provide an initialization file.

HAL9000: with this process you should also run into the one host-CPID for all cloned hosts issue as the host-CPID is generated at first startup and is based on the first available MAC. If you clone the VM after this was set all clones will use the same host-CPID. The Client can generate a new host-CPID if it detects a clash but that will only work if the first available MAC on the cloned VM changed after cloning from the original VM.

FritzBox360: I'll send you a PM with my skype username.

For some reason I was thinking the CPID was generated when a host first contacted a project since I hadn't had an issue.
So I started up 10 VMs from the same image and watched the process.

All hosts initially had the same CPID.
Attached hosts the same project.
All hosts still had the same CPID after project initialization.
My normal process it to check the project/hosts_user.php for the new hosts I am attaching to a project.
If I don't see them I issue an update request to BOINC

Sent project update request to hosts.
Most hosts generated new CPIDs.
Repeated updated request, twice more, until all new hosts were listed on project/hosts_user.php

I now see why I wasn't finding an issue with my process.
The tasks on the hosts that generated new CPID were marked as "Cancelled by server". Which is an error I generally ignore & may not even see as I don't necessarily check the stats of tasks everyday.

I found a workaround for my process.
I found setting <host_cpid></host_cpid> before starting BOINC on each new VM host was sufficient to generate a new CPID.
Which lets me avoid each new VM running a full BOINC initialization and benchmark before getting started.
ID: 77536 · Report as offensive
FritzBox360

Send message
Joined: 20 Apr 17
Posts: 4
Germany
Message 77548 - Posted: 24 Apr 2017, 7:42:10 UTC
Last modified: 24 Apr 2017, 7:42:25 UTC

Hi guys,
after a nice skype call from Christian we decided to change some points.

1 --> We need unique hostnames. --> We created a small script wich adds the 5th and 6th octet of the MAC from the host. This was possible because our OpenSLX virtualisation gives us a marginal
changed MAC from the host. So we have now all clients named with Ubuntu-XXxXX.

2 --> We created some initialization files for BOINC and the Project as mentioned here https://boinc.berkeley.edu/wiki/Initialization_files

3 --> We just clean installed BOINC but do not run it and we deleted the var/lib/boinc-client folder.

After some testing we need to change the startup behavior because BOINC was started to fast and had the old hostname. Then we had a few network performance issues related to our network.

But all in all - it should work now but just testet with 6 clients. But we will do some testing.

An easier way for changing the ID and joining the project would be nice. Maybe a trigger for BOINC to change or check the ID on next startup - depends on what the ID is created. Just a new MAC does not work. With a new MAC and a new hostname it worked.

We will try to do some more testing on that. Stay tuned.

Greetings from Mannheim, Germany
ID: 77548 · Report as offensive

Message boards : Questions and problems : Cloning a BOINC preinstalled VM

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.