Changes between Version 15 and Version 16 of CloudServer

Nov 16, 2015, 11:24:00 PM (5 years ago)



  • CloudServer

    v15 v16  
    1 = Deploying a BOINC server on the Amazon Elastic Computing Cloud =
    3 === What is Cloud Computing? ===
    5 Cloud Computing is a form of computation and storage that
    6 hides the complexity of hardware and software management
    7 from a user.  Typically, cloud computing services allow you
    8 to rent computing or storage services.  These services
    9 charge on a pay-per-use basis (roughly, about 0.10 per CPU
    10 hour or per GB/month).  Cloud computing services such as
    11 Amazon's [ Elastic Compute Cloud (EC2)] also allow one to
    12 tailor a computing environment through the use of virtual
    13 machines.  In addition, Amazon offers a [ Simple Storage Service (S3)]
    14 for enabling remote storage.
     1= Deploying a BOINC server in the Amazon Elastic Computing Cloud =
    163=== Why use a Cloud? === #why
    18 Hosting a BOINC server on a cloud is useful for the following reasons:
    19  * It's cheaper for small to medium projects to rent time on a cloud versus paying for hardware, bandwidth, and electric power on one's own.
    20  * It's easier and faster to use an existing OS image with the BOINC server already installed than to compile and configure it on one's own.
     5Hosting a BOINC server on a cloud has two advantages:
     6 * It's cheaper for small projects to rent time on a cloud versus paying for hardware, bandwidth, and electric power.
     7 * It's easier and faster to use an existing OS image with the BOINC server already installed than to compile and configure it.
    229For an in-depth cost-benefit analysis, please read:
    23 [ Cost-Benefit Analysis of Cloud Computing versus Desktop Grids].  Derrick Kondo, Bahman Javadi, Paul Malecot, Franck Cappello and David Anderson.  To appear in 18th International Heterogeneity in Computing Workshop, May 25 2009, Rome.
     10[ Cost-Benefit Analysis of Cloud Computing versus Desktop Grids].  Derrick Kondo, Bahman Javadi, Paul Malecot, Franck Cappello and David Anderson.  8th International Heterogeneity in Computing Workshop, May 25 2009, Rome.
    25 To calculate the cost-benefits for your own application, you can use this
    26 [ Excel spreadsheet].
    28 === How to use a Cloud for a BOINC server? ===
     13You can set up a free BOINC server in a few minutes using a preconfigured server for Amazon's Elastic Compute Cloud.
    30 We have created a custom virtual machine with a BOINC server
    31 installed and configured.  This includes all server software
    32 prerequisites and the correct permissions and accounts
    33 already setup.  All you need to do is to deploy this virtual
    34 machine image on the cloud and your server should be up and
    35 running.  This guide describes how to deploy this Amazon
    36 Machine Image (AMI) over Amazon's EC2 cloud. This can be
    37 done in minutes.  Testing this image deployement will cost you less then 0.50 USD so give it a try.
    39 === Prerequisites ===
    41 [ Create and setup an account with Amazon's EC2.]
    4316=== Deploying the BOINC Server Image ===
    45 Start the instance (ami-2290774b) with the BOINC server
    46 (stable 6.3.14 JAN 28 2009) installed on Debian Etch:
     20Navigate to to and create an account if needed (requires a credit card, but it will not be charged for 12 months). Click “EC2: Virtual Servers in the Cloud” and “Launch Instance”. Click “Community AMIs” (left sidebar), search for “BOINCServerTemplate”, and click on the blue Launch Button. Choose type “t2.micro: Free tier eligible” and “Review and Launch”. Click “Edit security groups”, “Add Rule” and select type “HTTP”. Click “Review and Launch” and then “Launch”. If you do not have a key pair, choose “Create a new key pair” and give it a name like BOINCKeypair, click on “Download Key Pair”, and save BOINCKeypair.pem in a safe place. Click on “Launch instance” and  “View Instances”. Wait until your instance changes from Instance State “Pending” to “running” with a green circle.
     22Paste the Public IP nnn.nnn.nnn.nnn (once it appears) into a web browser, and it should say “Apache2 Ubuntu Default Page”. You can assign a static IP address by choosing Elastic IP in the left side bar, allocate IP, and associate the IP with your instance. You can then buy a domain name and point to that static IP.
     26Then, connect to Public IP nnn.nnn.nnn.nnn by ssh. For instance, download PuTTY, use PuTTYgen to import the .pem file and save a .ppk file, and choose public ip address nnn.nnn.nnn.nnn, port 22, connection type SSH, choose Connection, SSH, Auth, and browse to find the .ppk file, click Open, click Yes at the “PuTTY Security Alert”. At the "login:" prompt, enter "ubuntu".
    49 ec2-run-instances ami-2290774b -k [key-pair]
     32$su boincadm
     34password: boincadm
     36$cd /home/boincadm/boinc/tools
     40ip-nnn-nnn-nnn-nnn  (note: this is the internal IP address, with dashes, differennt from the public IP)
     42$./make_project --url_base http://nnn.nnn.nnn.nnn/ --project_host ip-nnn-nnn-nnn-nnn --test_app uppercase
     46[Y/n] (hit enter)
     48$cd /home/boincadm/projects/uppercase/
     50$sudo -- sh -c 'cat /home/boincadm/projects/uppercase/uppercase.httpd.conf >> /etc/apache2/httpd.conf'
    52 Show the instance deployment status:
     58Check in a web browser that the page "http://nnn.nnn.nnn.nnn/uppercase" works and has the title "REPLACE WITH PROJECT NAME"
    55 ec2-describe-instances
    56 }}}
    58 After the instance status goes from pending to running
    59 (takes a few minutes), you
    60 should see the hostname of the form
    63 If you haven't already, open ports 22 and 80:
     68Choose "y" repeatedly
    66 ec2-authorize default -p 22
    67 ec2-authorize default -p 80
     76In a web browser, using the Public IP, navigate to "http://nnn.nnn.nnn.nnn/uppercase/server_status.php". Status should be "Running" green now for all Programs on the left.
    71 At this point, you should be able to access the web server
    72 at the URL:
    73 {{{}}}
    75 Then login to the instance:
     80Using a BOINC client, add the project "nnn.nnn.nnn.nnn/uppercase" and the client should start downloading and doing work. Congratulations you have a working BOINC project!
     84To protect your administrator page:
    78 ssh -i [private_key corresponding to key-pair]
     88$cd /home/boincadm/projects/uppercase/html/ops
     90$htpasswd -c .htpasswd boincadm
     92password: boincadm (choose your own password)
    81 As root,
     96In the web browser, navigate to your project administrator web interface at "http://nnn.nnn.nnn.nnn/uppercase_ops". Click on "Summary" to see work already done.
     100=== Creating Your Own Application ===
     104To create your application, you need a work generator, a client program in Windows/unix, and an assimilator to combine the work that comes back. To modify the work generator and sample assimilator:
    84 su boincadm
    85 cd
     108$cd /home/boincadm/boinc/sched/
     110$emacs sample_work_generator.cpp
     112$emacs sample_assimilator.cpp
     116$cp sample_work_generator /home/boinc/projects/uppercase/bin
     118$cp sample_assimilator /home/boinc/projects/uppercase/bin
     120$cd /home/boincadm/projects/uppercase
    88 In the home directory is a README.txt file with important
    89 information.
    91 === Setting Up a Test Project ===
    93 Here we describe how to create a test project using the
    94 uppercase application.
    96 Create a test project uppercase:
     130To modify the Unix client program in Unix,
    99 cd ~/boinc/tools
    100 ./make_project --url_base --db_passwd boincadmpw --test_app cplan
     134$emacs /home/boincadm/boinc/samples/example_app/uc2.cpp
     138$cd /home/boincadm/apps/example_app
     140$cp -r 22489 30000 (to create the directories for the new versions)
     142$cd 30000/x86_64-pc-linux-gnu
     144$cp /home/boinc/boinc/samples/example_app/uc2 /home/boinc/projects/apps/example_app/30000/x86_64-pc-linux-gnu/example_app_30000_x86_64-pc-linux-gnu
    103 Answer yes to all questions.
     148and delete the other files in that directory. To create updated client program for Windows, download Microsoft C++ Express Version 10, download the boinc source, Build All, in project "example_app" of the "boinc" solution, edit "uc2.cpp", Build All, run the compiled program win_build\Build\Win32\Release with input file "in", and review the output file "out" and the "stderr.txt". When the program is ready, use FileZilla to transfer the file example_app.exe to http://nnn.nnn.nnn.nnn/, SFTP, username "ubuntu", select the private .ppk key under Settings, SFTP, and click on the example_app.exe file in the left hand tree. Connect to your server, and look for the file in "/home/ubuntu". Then
    106 cd ~/projects/cplan
    107 ./bin/xadd
    108 ./bin/update_versions
    109 ./bin/start
     152$cp /home/ubuntu/example_app.exe /home/boincadm/projects/uppercase/apps/example_app/30000/windows_intelx86/example_app_30000_windows_intelx86.exe
    112 To check that all daemons are running:
     156and delete the older versions in that directory". Delete other architectures for which you do not have clients in the 30000 directory. Navigate in the browser to "http://nnn.nnn.nnn.nnn/uppercase_ops/", choose Manage application versions" and deprecated the old versions.
    114 {{{
    115 ./bin/status
    116 }}}
    118 Now any [ BOINC client] should be able to attach to the project and download
    119 workunits. For example, go to the project page ( and create a user account.
     160To fix the appearance of the page "http://nnn.nnn.nnn.nnn/uppercase", edit /home/boincadm/projects/uppercase/html/ especially to enter the project name in place of "REPLACE WITH PROJECT NAME".the "COPYRIGHT_HOLDER", the "SYS_ADMIN_EMAIL", and the smtp settings. To limit SPAM users, add akismet and recaptcha keys to config.xml.
    121 Then with a Linux command-line client in ~boincadm/BOINC, run
    122 {{{
    123 cd ~/BOINC
    124 ./boinc_client  -no_gui_rpc -attach_project [account_key]
    125 }}}
    127 At this point, the client should attached to the project and begin downloading work.
     164=== Notes ===
    129 === Technical Notes ===
    131 Below is not required reading, but we describe how and why
    132 the image was built.
     168Change the following default passwords: for the Unix user boincadm password boincadm, and for the mysql root user password boincadm. The mysql user boincadm has not password; add one and update config.xml. Edit /etc/apache2/apache2.config and /etc/apache2/httpd.config to use the "Require" directory in both places.
    134 ==== BOINC Configuration ====
    136 You'll find in the ~boincadm directory :
    138  * BOINC: directory with client for testing purposes
    139  * projects:  directory where BOINC projects are installed
    140  * boinc_server_stable_r16906_6.3.14: directory with server source code
    144 ==== Why Xen? ====
    147 The general approach was to build a Debian EC2 AMI from a
    148 Xen image.  (The alternative approach was to use an existing
    149 Debian AMI.)  However, we chose the Xen approach primarily
    150 to be able to install and test things without having to
    151 worry about EC2 hourly costs. As a side-effect, one can use
    152 the Xen image either as a standalone server or over EC2 (as
    153 it's Xen based).
    155 These links describe the setup involved:
    156 [ Debian from Xen Images],
    157 [ Bundle Debian AMI using Ubuntu]
    160 If you would like to use the Xen image directly, you can
    161 download the Xen image
    162 [ disk.img.tgz (1.5GB)] and
    163 [ swap.img.tgz (.5MB)],
    164 and config file
    165 [ boinc_debian_etch.cfg]
    166  and go from there.
    168 The Xen image above was configured to use a bridged Ethernet connection and DHCP.
    170 ==== Hostname Configuration ====
    172 The hostname can change each time an instance is started.
    173 As such, the start-up script {{{/etc/rc.local}}} was modified to query
    174 Amazon for the hostname and to set it at each boot.
    177 ==== IP Address ====
    179 The IP address could change each time the instance is
    180 created.  To associate a fixed address (regardless of the
    181 instance), one should use
    182 [  Amazon's Elastic IP Addresses].
    184 In addition, one can configure the image to use dynamic DNS
    185 to associate a domain name with the instance. See the following links.
    186 [ Dynamic DNS with ZoneEdit]
    187 [ Dynamic DNS Setup]
    189 ==== SSH ====
    191 SSH server host keys are (re)generated each time the instance is booted in {{{/etc/rc.local}}}.  This is to help prevent man-in-the-middle-attacks.
    193 {{{rc.local}}} is configured to randomly change the root
    194 password at each boot, and to download your ssh public key
    195 and cat it to the authorized_keys file to allow remote passwordless access.
    197 To verify the identity of the host before logging in, you can run {{{ec2-get-console-output}}} to get the key fingerprint written to the console during the bootup process.  The key fingerprints will look something like:
    198 {{{
    199 Restarting OpenBSD Secure Shell server: sshd.
    200 ------- KEY FINGERPRINTS -------
    201 2048 28:f1:0c:51:95:82:c6:a1:a2:58:a3:37:7f:39:ef:f9 /etc/ssh/
    202 1024 90:f2:98:4b:e6:c5:c7:7a:23:6e:ef:67:d0:be:72:a9 /etc/ssh/
    203 --------------------------------
    204 }}}
    205 during bootup.
    206 Then you can compare this key fingerprint with the one reported by ssh when you log in to verify the host identity.
    208 === To do ===
    210  * Consider setting up dynamic DNS by default in the bundle
    212172=== Questions? ===
    214 Email [ Derrick Kondo] (derrick.kondo :: inria fr)
     176Email: daniel-monroe :: verizon net.