Changes between Version 64 and Version 65 of ServerIntro


Ignore:
Timestamp:
Jun 20, 2012, 11:43:32 AM (12 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • ServerIntro

    v64 v65  
    33= Setting up a BOINC server = #ServerIntro
    44== Hardware ==
    5 For experimentation and debugging, you can use almost any computer as a BOINC server. Before deploying a project more widely, make sure that your server has adequate performance, availability, and security. Some factors:
    6 
    7  * Your Internet connection should have adequate performance and reliability.  Your server must have a static IP address.
    8  * Your server should have good CPU speed(dual Xeon or Opteron), at least 2 GB of RAM, and at least 40 GB of free disk space.  For a high-traffic project, use a machine with 8 GB of RAM or more, and 64-bit processors.
    9  * Do whatever you can to make it highly reliable (UPS power supply, RAID disk configuration, hot-swappable spares, temperature-controlled machine room, etc.).
    10  * Put it behind a firewall; allow access via port 80 (HTTP) and optionally 443 (HTTPS).
    11  * Make it secure; turn off any unneeded network services, especially those that use plaintext passwords (like FTP or Telnet).
     5For experimentation and debugging,
     6you can use almost any computer as a BOINC server.
     7Before deploying a project more widely,
     8make sure that your server has adequate performance, availability, and security.
     9Some factors:
     10
     11 * Your Internet connection should have adequate performance and reliability.
     12   Your server must have a static IP address.
     13 * Your server should have good CPU speed(dual Xeon or Opteron),
     14   at least 2 GB of RAM, and at least 40 GB of free disk space.
     15   For a high-traffic project, use a machine with 8 GB of RAM or more,
     16   and 64-bit processors.
     17 * Do whatever you can to make it highly reliable (UPS power supply,
     18   RAID disk configuration, hot-swappable spares,
     19   temperature-controlled machine room, etc.).
     20 * Put it behind a firewall; allow access via port 80 (HTTP)
     21   and optionally 443 (HTTPS).
     22 * Make it secure; turn off any unneeded network services,
     23   especially those that use plaintext passwords (like FTP or Telnet).
    1224
    1325Info on increasing capacity and reliability is [wiki:MultiHost here].
    1426
    15 Another approach - which eliminates both hardware and software issues - is to deploy a BOINC server on the [wiki:CloudServer Amazon Elastic Computing Cloud].
     27Another approach - which eliminates both hardware and software issues -
     28is to deploy a BOINC server on the
     29[wiki:CloudServer Amazon Elastic Computing Cloud].
    1630
    1731== Software ==
    1832There are several ways to deploy the BOINC server software:
    1933
    20  * The easiest way to set up a BOINC server is to use a [wiki:VmServer BOINC server VM] that we've created for that purpose, with all the necessary software already installed. You can run this virtual machine using the [http://www.vmware.com/download/player/ VMWare player] on any Intel-based computer (Windows, Linux, or Mac OS X).  If you take this approach, skip the rest of this page.
    21 
    22  * Debian and Ubuntu offer the boinc-server-maker package to create BOINC project more easily on local or remote machines. This ongoing effort, described on their [http://wiki.debian.org/BOINC/ServerGuide Wiki] aims at full compatibility with the BOINC Virtual Machine. If anyone is interested in helping maintaining a boinc-server package for Debian, please contact David Anderson. Again, if you take this approach, skip the rest of this page.
    23 
    24  * You can use any Unix system as a BOINC server; we recommend using a recent Linux release. Disable SELinux. General instructions are given below, followed by a [#cookbook-debian40 a cookbook for setting up a BOINC server on a Debian 4.0 system].
     34 * The easiest way to set up a BOINC server is to use a
     35  [wiki:VmServer BOINC server VM] that we've created for that purpose,
     36  with all the necessary software already installed.
     37  You can run this virtual machine using the
     38  [http://www.vmware.com/download/player/ VMWare player]
     39  on any Intel-based computer (Windows, Linux, or Mac OS X).
     40  If you take this approach, skip the rest of this page.
     41
     42 * Debian and Ubuntu offer the boinc-server-maker package to create
     43   BOINC project more easily on local or remote machines.
     44   This ongoing effort, described on their
     45   [http://wiki.debian.org/BOINC/ServerGuide Wiki] aims at full compatibility
     46   with the BOINC Virtual Machine.
     47   If anyone is interested in helping maintaining a boinc-server package
     48   for Debian, please contact David Anderson.
     49   Again, if you take this approach, skip the rest of this page.
     50
     51 * You can use any Unix system as a BOINC server;
     52   we recommend using a recent Linux release.
     53   Disable SELinux. General instructions are given below,
     54   followed by a [#cookbook-debian40 a cookbook for setting up
     55   a BOINC server on a Debian 4.0 system].
    2556
    2657== Installing the BOINC server on Unix == #general
    … …  
    2859BOINC server programs run as two different users:
    2960
    30  * The scheduler and file upload handler are CGI programs, so they run as the same user as the web server (on Fedora this is user '`apache`'; on Debian it's '`www-data`').
    31  * BOINC daemons run as whoever created the project (let's say user '`boincadm`', group '`boinc`'). '''Do not run BOINC daemons (or anything else) as root'''.
    32 
    33 By default, the directories created by user `apache` are not world-writable. This causes problems: for example, when the file upload handler creates a directory in the [wiki:DirHierarchy upload hierarchy], it's owned by (`apache`, `apache`), and the [wiki:FileDeleter file deleter] (which runs as `boincadm`) won't be able to delete the files there.
    34 
    35 To solve this problem, edit `/etc/group` so that `apache` belongs to group `boinc`, i.e. the line:
     61 * The scheduler and file upload handler are CGI programs,
     62   so they run as the same user as the web server
     63   (on Fedora this is user '`apache`'; on Debian it's '`www-data`').
     64 * BOINC daemons run as whoever created the project
     65   (let's say user '`boincadm`', group '`boinc`').
     66   '''Do not run BOINC daemons (or anything else) as root'''.
     67
     68By default, the directories created by user `apache` are not world-writable.
     69This causes problems: for example, when the file upload handler creates
     70a directory in the [wiki:DirHierarchy upload hierarchy],
     71it's owned by (`apache`, `apache`), and the [wiki:FileDeleter file deleter]
     72(which runs as `boincadm`) won't be able to delete the files there.
     73
     74To solve this problem, edit `/etc/group` so that `apache` belongs
     75to group `boinc`, i.e. the line:
    3676
    3777{{{
    … …  
    4585(Apache will need to be stopped/restarted for this to take effect.)
    4686
    47 Both `boincadm` and `apache` should have [http://www.slackwiki.org/Permissions_and_Umasks umasks] that allow group read and write.
    48 
    49 {{{
    50 #!comment That umask thing needs to be clarified; I had to go see an old version of GroupPermissions to see how umasks were set. Umasks are "inverted", that makes them hard to understand (they mark permissions to UNset).
    51 }}}
    52 When you create a BOINC project using [wiki:MakeProject make_project], the critical directories are owned by `boincadm` and have the set-GID bit set; this means that any directories or files created by `apache` in those directories will have group `boinc` (not group `apache`). The BOINC software makes all directories group read/write. Thus, both `apache` and `boinc` will have read/write access to all directories and files, but other users will have no access.
     87Both `boincadm` and `apache` should have
     88[http://www.slackwiki.org/Permissions_and_Umasks umasks]
     89that allow group read and write.
     90
     91{{{
     92#!comment That umask thing needs to be clarified;
     93I had to go see an old version of GroupPermissions to see how umasks were set.
     94Umasks are "inverted", that makes them hard to understand
     95(they mark permissions to UNset).
     96}}}
     97When you create a BOINC project using [wiki:MakeProject make_project],
     98the critical directories are owned by `boincadm` and have the set-GID bit set;
     99this means that any directories or files created by `apache`
     100in those directories will have group `boinc` (not group `apache`).
     101The BOINC software makes all directories group read/write.
     102Thus, both `apache` and `boinc` will have read/write access
     103to all directories and files, but other users will have no access.
    53104
    54105To fix permissions on an existing project, do:
    … …  
    62113chmod 02770 html/user_profile
    63114}}}
    64 You may also need to change the ownership of these directories and all their subdirectories to `boincadm/boinc`.  If you're running several projects on the same server and want to isolate them from each other, you can create a different user and group for each project, and add `apache` to all of the groups.
    65 
    66 When serving your project files from Apache, note that all directories up to and including the `html` directory must have execute permissions.  For example, if you use `make_project` to create the project template in your home directory, your home directory must have 711 permissions as opposed to the default of 700. If this is not corrected, you will receive a '''403 Forbidden''' error when attempted to browse to your project page. See [http://httpd.apache.org/docs/1.3/misc/FAQ.html#forbidden more information on dealing with Apache permissions problems].
     115You may also need to change the ownership of these directories
     116and all their subdirectories to `boincadm/boinc`.
     117If you're running several projects on the same server
     118and want to isolate them from each other,
     119you can create a different user and group for each project,
     120and add `apache` to all of the groups.
     121
     122When serving your project files from Apache,
     123note that all directories up to and including the `html` directory
     124must have execute permissions.
     125For example, if you use `make_project` to create the project template
     126in your home directory,
     127your home directory must have 711 permissions
     128as opposed to the default of 700.
     129If this is not corrected, you will receive a '''403 Forbidden''' error
     130when attempted to browse to your project page.
     131See [http://httpd.apache.org/docs/1.3/misc/FAQ.html#forbidden
     132more information on dealing with Apache permissions problems].
    67133
    68134=== Installing BOINC software === #installing
    69  * Download and install the needed [wiki:SoftwarePrereqsUnix software prerequisites].
     135 * Download and install the needed
     136   [wiki:SoftwarePrereqsUnix software prerequisites].
    70137 * [wiki:SourceCode Download] the BOINC software.
    71138 * [wiki:BuildSystem Configure and build] the BOINC software.
    72139
    73140=== Operating system configuration === #OS-config
    74 Some parts of the BOINC server (the feeder and scheduling server) use shared memory. Hosts where these run must have shared memory enabled, with a maximum segment size of at least 32 MB. How to do this depends on the operating system; read [http://developer.postgresql.org/docs/postgres/kernel-resources.html Managing Kernel Resources on PostgreSQL manual] for more information.
     141Some parts of the BOINC server (the feeder and scheduling server)
     142use shared memory.
     143Hosts where these run must have shared memory enabled,
     144with a maximum segment size of at least 32 MB.
     145How to do this depends on the operating system;
     146read [http://developer.postgresql.org/docs/postgres/kernel-resources.html
     147Managing Kernel Resources on PostgreSQL manual] for more information.
    75148
    76149=== MySQL notes === #MySQL-notes
    77  * After installing and running the server, grant permissions for your own account and for the account under which Apache runs ('nobody' in the following; may be different on your machine). All MySQL accounts should be password protected, especially `root`.
     150 * After installing and running the server,
     151   grant permissions for your own account and for the account under
     152   which Apache runs ('nobody' in the following;
     153   may be different on your machine).
     154   All MySQL accounts should be password protected, especially `root`.
    78155{{{
    79156mysql -u root
    … …  
    83160grant all on *.* to nobody identified by 'password';
    84161}}}
    85  * Set your `PATH` environment variable to include MySQL programs (typically `/usr/local/mysql` and `/usr/local/mysql/bin`).
    86  * You'll need to back up your database. Generally this requires stopping the project, making a copy or snapshot, and restarting. An example is [//mysql_backup.txt here].
    87  * BOINC gets MySQL compiler and linker flags from a program called `mysql_config` which comes with your MySQL distribution. This sometimes references libraries that are not part of your base system installation, such as `-lnsl` or `-lnss_files`. You may need to install additional packages (often you can use something called 'mysql-dev' or 'mysql-devel') or fiddle with Makefiles.
    88  * MySQL can be the bottleneck in a BOINC server. To optimize its performance, read about [wiki:MysqlConfig configuring MySQL for BOINC].
     162 * Set your `PATH` environment variable to include MySQL programs
     163   (typically `/usr/local/mysql` and `/usr/local/mysql/bin`).
     164 * You'll need to back up your database.
     165   Generally this requires stopping the project,
     166   making a copy or snapshot, and restarting.
     167   An example is [//mysql_backup.txt here].
     168 * BOINC gets MySQL compiler and linker flags from a program called
     169   `mysql_config` which comes with your MySQL distribution.
     170   This sometimes references libraries that are not part of your base system
     171   installation, such as `-lnsl` or `-lnss_files`.
     172   You may need to install additional packages
     173   (often you can use something called 'mysql-dev' or 'mysql-devel')
     174   or fiddle with Makefiles.
     175 * MySQL can be the bottleneck in a BOINC server.
     176   To optimize its performance,
     177   read about [wiki:MysqlConfig configuring MySQL for BOINC].
    89178 * [//mysql_cluster.txt Notes on running MySQL on a cluster].
    90179
    … …  
    94183
    95184=== Apache notes === #apache-notes
    96 In `httpd.conf`, set the default MIME type as follows (otherwise you'll get file upload signature verification errors):
     185In `httpd.conf`, set the default MIME type as follows
     186(otherwise you'll get file upload signature verification errors):
    97187
    98188{{{
    99189DefaultType application/octet-stream
    100190}}}
    101 To limit denial-of-service attacks, we recommend turning off directory indexing by adding `-Indexes` to the [http://httpd.apache.org/docs/2.0/mod/core.html#options Options] directive.
    102 
    103 Make sure that the filename extensions used by your application or data files aren't handled in undesired way by Apache. For example, if some of your files have a '.map' extension, remove the line
     191To limit denial-of-service attacks, we recommend turning off directory
     192indexing by adding `-Indexes` to the
     193[http://httpd.apache.org/docs/2.0/mod/core.html#options Options] directive.
     194
     195Make sure that the filename extensions used by your application or data files
     196aren't handled in undesired way by Apache.
     197For example, if some of your files have a '.map' extension, remove the line
    104198
    105199{{{
    … …  
    108202from httpd.conf
    109203
    110 Apache has a default request size limit of 1 MB. If your project is likely to exceed this (large output files, large scheduler request messages) increase this, e.g. to 128 MB:
     204Apache has a default request size limit of 1 MB.
     205If your project is likely to exceed this
     206(large output files, large scheduler request messages) increase this,
     207e.g. to 128 MB:
    111208
    112209{{{
    … …  
    115212}}}
    116213=== PHP notes === #PHP-notes
    117  * By default, BOINC uses PHP's `mail` function to send email to participants. This uses sendmail. If this doesn't work, you can use [http://phpmailer.sourceforge.net/ PHPMailer] instead, which is a very flexible mail-sending mechanism. To do this:
    118    * Download PHPMailer and put it under PROJECT/html/inc/phpmailer (i.e. the files class.smtp.php and class.phpmailer.php should be in that directory).
    119    * Set the following variables in your PROJECT/html/project/project.inc file (substitute your own values):
     214 * By default, BOINC uses PHP's `mail` function to send email to participants.
     215  This uses sendmail.
     216  If this doesn't work, you can use
     217  [http://phpmailer.sourceforge.net/ PHPMailer] instead,
     218  which is a very flexible mail-sending mechanism. To do this:
     219   * Download PHPMailer and put it under PROJECT/html/inc/phpmailer
     220     (i.e. the files class.smtp.php and class.phpmailer.php should be
     221         in that directory).
     222   * Set the following variables in your PROJECT/html/project/project.inc file
     223     (substitute your own values):
    120224{{{
    121225$USE_PHPMAILER = true;
    … …  
    125229
    126230=== Common problems === #common-problems
    127 After adding a new user, you have to check if the home directory was created. Sometimes you have to manually create it.
     231After adding a new user, you have to check if the home directory was created.
     232Sometimes you have to manually create it.
    128233
    129234{{{
    … …  
    131236$ chown {username}:{username} /home/{username}
    132237}}}
    133 The link /usr/bin/gcc that points to the currently used compiler could be missing. This could lead to errors when running the make command. Especially when trying to compile the graphical part of the API you need the /usr/bin/g++ link (make gives a warning that it couldn't find GL/GLU/GLUT although it is installed). So you have to create the symbolic links, depending on your gcc and g++ version, first.
     238The link /usr/bin/gcc that points to the currently used compiler
     239could be missing.
     240This could lead to errors when running the make command.
     241Especially when trying to compile the graphical part of the API
     242you need the /usr/bin/g++ link
     243(make gives a warning that it couldn't find GL/GLU/GLUT
     244although it is installed).
     245So you have to create the symbolic links,
     246depending on your gcc and g++ version, first.
    134247
    135248{{{
    … …  
    137250$ ln -s /usr/bin/g++-4.11 /usr/bin/g++
    138251}}}
    139 At newer linux distributions you can’t login as user root. Instead there is a user notroot with whom you can login, but you won’t have root privileges. If an application or script needs root privileges, you are forced to enter the password you entered during the installation. In the console window you can get root privileges with the su (Debian) or sudo (Ubuntu, Fedora) command also. Note the single quotes around the command when using su -c !
     252At newer linux distributions you can’t login as user root.
     253Instead there is a user notroot with whom you can login,
     254but you won’t have root privileges.
     255If an application or script needs root privileges,
     256you are forced to enter the password you entered during the installation.
     257In the console window you can get root privileges with the
     258su (Debian) or sudo (Ubuntu, Fedora) command also.
     259Note the single quotes around the command when using su -c !
    140260
    141261{{{
    … …  
    143263$ sudo {command}
    144264}}}
    145 If you have problems executing the sudo command, you have to add the user to the /etc/sudoers file. To edit this file just use this command:
     265If you have problems executing the sudo command,
     266you have to add the user to the /etc/sudoers file.
     267To edit this file just use this command:
    146268
    147269{{{
    148270$ sudo visudo
    149271}}}
    150 In this file you have to add another line after the line for notroot. You can use this:When running Wheezy or later, you find all
     272In this file you have to add another line after the line for notroot.
     273You can use this:When running Wheezy or later, you find all
    151274
    152275{{{
    … …  
    155278== Cookbook for Debian 4.0 == #cookbook-debian40
    156279
    157 Please follow these recipes only for your basic orientation. The Debian landscape has changed a lot over since Debian 4.0. You can inspect the build and runtime dependencies of the boinc-server-maker package [http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob_plain;f=debian/control;hb=HEAD here] and the latter more graphically also [http://packages.debian.org/unstable/net/boinc-server-maker here].
     280Please follow these recipes only for your basic orientation.
     281The Debian landscape has changed a lot over since Debian 4.0.
     282You can inspect the build and runtime dependencies of the
     283boinc-server-maker package
     284[http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob_plain;f=debian/control;hb=HEAD here]
     285and the latter more graphically also
     286[http://packages.debian.org/unstable/net/boinc-server-maker here].
    158287=== Install software prerequisites === #cookbook-prereq
    159 Common packages for building BOINC (based on [wiki:SoftwarePrereqsUnix Software prerequisites (Unix/Linux)]):
     288Common packages for building BOINC (based on
     289[wiki:SoftwarePrereqsUnix Software prerequisites (Unix/Linux)]):
    160290
    161291{{{
    … …  
    217347xserver-org
    218348}}}
    219 This list contains all necessary packages for the BOINC server and the client components. At first startup, these packages should be installed with aptitude. The graphical desktop manager is optional, but makes the configuration of the server a bit easier. The list contains no security packages to secure your server! Please ask your system administrator or read a good Linux book on how to secure your server against attacks.
     349This list contains all necessary packages for the BOINC server and the client
     350components.
     351At first startup, these packages should be installed with aptitude.
     352The graphical desktop manager is optional,
     353but makes the configuration of the server a bit easier.
     354The list contains no security packages to secure your server!
     355Please ask your system administrator or read a good Linux book
     356on how to secure your server against attacks.
    220357
    221358Hint: You can search packages within aptitude by pressing the '/' key.
    … …  
    226363'''With KDE:'''
    227364
    228   Go to ‘K-Menu’ -> ‘Settings’ -> ‘System Administration’ -> ‘User Management’, claim administrative privileges
    229 
    230   Create a new user, for example: boincadm
    231 
    232   After that go to the ‘Groups’ tab and add the user ‘www-data’ to the group ‘boincadm’.
     365 *  Go to ‘K-Menu’ -> ‘Settings’ -> ‘System Administration’ -> ‘User Management’,
     366   claim administrative privileges
     367
     368 *  Create a new user, for example: boincadm
     369
     370 *  After that go to the ‘Groups’ tab and add the user ‘www-data’
     371  to the group ‘boincadm’.
    233372
    234373'''Without a Graphical Desktop Manager:'''
    … …  
    241380
    242381=== Configuration of the MySQL server === #cookbook-mysql
    243 If you're using KDE: open a new console window (’K-menu’ -> ‘Debian’ -> ‘XShells’ -> ‘Konsole’).
     382If you're using KDE: open a new console window
     383(’K-menu’ -> ‘Debian’ -> ‘XShells’ -> ‘Konsole’).
    244384
    245385Define a new MySQL root password.
    … …  
    255395> SET PASSWORD FOR 'boincadm'@'localhost'='';
    256396}}}
    257 The permissions can be limited to project database only, later; Defining an empty password simplifies the installation, can be changed later.
     397The permissions can be limited to project database only, later;
     398Defining an empty password simplifies the installation, can be changed later.
    258399
    259400=== Download the BOINC source code === #cookbook-download-code
    … …  
    276417$ svn co http://boinc.berkeley.edu/svn/tags/boinc_core_release_5_9_5/boinc boinc_595
    277418}}}
    278 Hint: Normally you won't need a self-compiled client, you can always get the latest one from [//download_all.php BOINC website].
    279 
    280 NOTE: Tags are only for client software, always checkout the latest revision for the server!
     419Hint: Normally you won't need a self-compiled client,
     420you can always get the latest one from [//download_all.php BOINC website].
     421
     422NOTE: Tags are only for client software,
     423always checkout the latest revision for the server!
    281424
    282425=== Compiling BOINC source code === #cookbook-compiling