[[PageOutline]]

= Running apps in VirtualBox virtual machines  =

== Introduction ==

BOINC supports applications that run in !VirtualBox virtual machines.
This provides two benefits:
 * You don't need to build app versions for different architectures.
   Develop your app in your environment of choice (say, Debian Linux),
   and then bundle the resulting executable together with
   a virtual machine image containing an appropriate runtime environment.
   The application can then be run on all platforms
   (Windows, Mac OS X, all versions of Linux)
   with no additional work on your part.
 * Virtual machines provide the strongest available security sandbox;
   a program running in a virtual machine cannot access or modify
   the host system.
   This makes it feasible to deploy untrusted applications.

BOINC's support for VM apps is based on a program called '''vboxwrapper'''
that interfaces between the BOINC client and the !VirtualBox system.

=== Restrictions ===

 * Commercial operating systems like Windows and Mac OS X
  have a license with a pay-per-user clause,
  so in general you can't use them in the VM image.
  Similarly, you can't include pay-per-user software such as Matlab in the VM image.

 * !VirtualBox runs only on Intel-compatible processors.
  If you want to support other processors (such as ARM, SPARC, etc.),
  you'll need to use non-VM-based app versions.

 * Currently you can't run GPU applications in !VirtualBox VMs.
  This may change in the future.

=== 32/64 bit issues ===

A VM image is called 32- or 64-bit depending on the operating system it contains.
The BOINC host population includes 32-bit and 64-bit hosts.
64-bit hosts can run 32-bit VMs, but not conversely.
You can choose to provide 32- or 64-bit VM images, or both.

Possible reasons for using 64-bit VM images:
 * The 64-bit version of your app runs significantly faster than the 32-bit version.
 * Your app uses more than 3 GB of virtual address space.

If you provide only 32-bit VM images,
you must still create separate 32- and 64-bit app versions,
using the same VM image but different '''vboxwrapper''' executables.
('''vboxwrapper''' is a C++ program and has different executables
for each platform, include 32- and 64-bit; these are available below.
The 32-bit vboxwrapper will generally not work on a 64-bit machine).

=== Packaging options ===

There are two ways to package VM apps.
NOTE: in the following, we use '''application''' and '''application version'''
with their [AppVersion BOINC-specific meanings];
we'll use '''executable''' to refer to the program that runs within the VM.

 * '''Single-purpose app''':  Include the executable with the application version.
  Create a separate application for each executable you want to run.
 * '''Multi-purpose app''': Include the executable in each workunit.
  This allows you to use a single application for as many executables as you like.
  In this case, consider making the executable file
  [JobSubmission sticky]; that way, clients will download it only once.
  
== Creating app versions ==

You must [AppVersionNew create app versions] for each platform you want to support;
the app versions differ in which '''vboxwrapper''' executable they use.
If you use both 32- and 64-bit VMs,
the versions will also differ in the VM image and the application executable.

The application versions for a given platform are of plan class "vbox32"
(for 32-bit machines) or "vbox64" (for 64-bit machines).

To enable multiple cores use the plan classes "vbox32_mt" and "vbox64_mt".
By default it will cause the server to assign 2 threads(virtual cores) per VM task.
This behavior can be modified in sched_customize.cpp.

For single-purpose apps, an app version includes the following files:

 * The VM image, in !VirtualBox format.
  * Must have the '''copy_file''' attribute.
  * Should have the '''gzip''' attribute for faster download to 7.0+ clients.
  * Must have logical name "vm_image.vdi".
 * The application executable to be run in the VM image.
  * This may be a shell script or a binary program.
  * The logical name must be '''shared/boinc_app'''.
 * Other files needed by the application, all with logical names starting with '''shared/'''.
 * An XML '''job description file''' with logical name '''vbox_job.xml''' (see below)
 * '''vboxwrapper''', compiled for the platform (executables are available below).
 * All scripts and executables must have the execute permission set.

For multi-purpose apps, any of these files except '''vboxwrapper''' may
be included in the workunit instead of the app version.

Include '''<dont_throttle/>''' in the version.xml file; !VirtualBox does its own CPU throttling.

Typically you can use the same VM image for multiple applications.
This reduces network traffic and client disk usage.

=== The job description file ===

The job description file has logical name '''vbox_job.xml'''
(its physical name should include a version number and other info).
It has following structure:

{{{
<vbox_job>
   <os_name>name</os_name>
   <memory_size_mb>N</memory_size_mb>
   [ <vm_disk_controller_type>ide|sata|scsi|floppy|sas</vm_disk_controller_type> ]
   [ <vm_disk_controller_model>LSILogic|LSILogicSAS|BusLogic|IntelAHCI|PIIX3|PIIX4|ICH6|I82078</vm_disk_controller_model> ]
   [ <enable_network/> ]
   [ <enable_shared_directory/> ]
   [ <enable_floppyio>0|1</enable_floppyio> ]
   [ <enable_remotedesktop>0|1</enable_remotedesktop> ]
   [ <job_duration>X</job_duration> ]
   [ <fraction_done_filename>filename</fraction_done_filename> ]
   [ <pf_guest_port>N</pf_guest_port> ]
   [ <pf_host_port>N</pf_host_port> ]
   [ <copy_to_shared>file</copy_to_shared> ]
</vbox_job>
}}}
The elements are:

 '''os_name''':: the name of the guest OS as defined by !VirtualBox,
   e.g. "Linux26", "Linux26_64", "Linux24", etc.
   To see a list of all available OS names, install !VirtualBox on a system,
   and type "vboxmanage list ostypes".
 '''memory_size_mb''':: the amount of physical memory allocated to the VM, in megabytes.
 '''vm_disk_controller_type''':: which disk controller type to emulate.
 '''vm_disk_controller_model''':: which disk controller model to emulate.
 '''enable_network''':: if present, allow the VM to do network access.
 '''enable_shared_directory''':: if present, create a directory that is shared
   between the host OS and the guest OS.
   Must be set if your application has input or output files.
 '''enable_floppyio''':: create a floppy disk image in the VM,
  containing the contents of init_data.xml.
 '''enable_remotedesktop''':: If the Oracle VirtualBox Extension are installed, it'll enable
  the use of a remote desktop client to view the console of the VM.
 '''job_duration''':: this specifies the maximum elapsed time of the job,
  after which vboxwrapper will kill the VM and exit normally.
 '''fraction_done_filename''':: the name of a file to which the app will
  periodically write its fraction done (0 to 1).
  This is used by the wrapper to report overall fraction done.
 '''pf_guest_port''':: enable port forwarding to port N within the VM.
  This is assumed to be a web server providing application graphics.
 '''pf_host_port''':: host port # for port forwarding.
  If missing or zero, the host port number is assigned dynamically, and is written to a file
  '''vbox_port_forward.xml''' in the slot directory.
 '''copy_to_shared''':: file(s) you wish to copy into the shared directory before launch.
  This directive can be used more than once.

== Creating jobs for VM apps ==

The input and output files of a VM app must

 * Have logical names starting with '''shared/'''.
 * Have the '''copy_file''' attribute.

This causes the BOINC client to copy them to and from the '''slot/x/shared/''' directory.

== Premade vboxwrapper executables ==

Windows:

x86: [http://boinc.berkeley.edu/dl/vboxwrapper_26032_windows_intelx86.zip vboxwrapper_26032_windows_intelx86.zip]

x64: [http://boinc.berkeley.edu/dl/vboxwrapper_26032_windows_x86_64.zip vboxwrapper_26032_windows_x86_64.zip]

Mac OS X:

x86: [http://boinc.berkeley.edu/dl/vboxwrapper_26032_i686-apple-darwin.zip vboxwrapper_26032_i686-apple-darwin.zip]

x64: [http://boinc.berkeley.edu/dl/vboxwrapper_26029_x86_64-apple-darwin.zip vboxwrapper_26032_x86_64-apple-darwin.zip]

Linux:

x86: [http://boinc.berkeley.edu/dl/vboxwrapper_26032_i686-pc-linux-gnu.zip vboxwrapper_26032_i686-pc-linux-gnu.zip]

x64: [http://boinc.berkeley.edu/dl/vboxwrapper_26032_x86_64-pc-linux-gnu.zip vboxwrapper_26032_x86_64-pc-linux-gnu.zip]

== Premade Linux VM Images ==

These VM images were built using the above instructions for creating VM images.
They contain Debian 4.0, without GCC or any build tools installed.
They contain the example startup script.

x86: [http://boinc.berkeley.edu/dl/vmimage_x86.zip vmimage_x86.zip]

x64: [http://boinc.berkeley.edu/dl/vmimage_x64.zip vmimage_x64.zip]

In most cases, you can use these VM images with no modifications.
If your application uses libraries not on the VM images,
you can add them as follows:

 * Run !VirtualBox, and open the VM image
 * Hit CTRL-C when see "BOINC VM starting" in the console window
 * Install whatever you want (can use '''apt-get install''' to
   install Debian packages).
 * when you're done, type
{{{
shutdown -hP 0
}}}

The VM image now has the additional libraries.
Rename it to avoid confusion with the original version.

== Creating your own VM images ==
=== Requirements of the VM ===

The VM, when booted, must do the following:

 * If the applications has input or output files, mount the shared directory using

{{{
mount -t vboxsf shared /root/shared
}}}
 where `/root/shared` is the path where the shared directory is to be mounted.
 In this case the VM must contain the !VirtualBox "guest additions".
 Guest additions are required for shared folders to work.

 * Run the application.

 * When the application is finished, shut down the VM
  (e.g., by running '''shutdown''' on Linux).

These steps are typically done by a '''startup script''' in the VM image.
An example startup script is given below.
This script runs the application by doing the following:
 * cd into the shared directory
 * execute '''boinc_app''', and wait for it to exit.

Using this script,
your application executable must have logical name '''share/boinc_app'''.

Doing things this way, the VM image is independent of the application.
You can use the a single VM image for many applications.

'''Attention:''' If your '''boinc_app''' is a bash or perl script you may get problems when the VM is restored from a snapshot. To circumvent this you have to change your startup script to copy the contents of the shared/ directory to another directory 'inside' the VM and execute it there. For example:
{{{
echo -- Launching boinc_app
if [ -f /root/shared/boinc_app ]; then
    mkdir /root/worker
    cp -r /root/shared/* /root/worker/
    cd /root/worker/
    ./boinc_app
    cd /root/
    rm -rf /root/worker
    shutdown -hP 0
else
}}}
This way you can still reuse the VM for other applications but have to make sure that your '''boinc_app''' control script is copying the output files of the application to ´/root/shared/´ before exiting. This may take some time, so you should do something like:
{{{
cp outfile1.zip /root/shared/out1.zip.tmp
cp outfile2.zip /root/shared/out2.zip.tmp
{...}
mv /root/shared/out1.zip.tmp /root/shared/out1.zip
mv /root/shared/out2.zip.tmp /root/shared/out2.zip
}}}

=== Example startup script ===

The example startup script follows.
You can deploy it by appending to '''/root/.bashrc''' in the VM image.
{{{
echo --- BOINC VM starting
sleep 5
}}}

The "sleep 5" gives you time to break into a console session via CTRL-C
if you need to make changes to the VM in the future.

{{{
echo --- Mounting shared directory
mount -t vboxsf shared /root/shared
if [ $? -ne 0 ]; then
    echo --- Failed to mount shared directory
    sleep 5
    shutdown -hP 0
fi

echo -- Launching boinc_app
if [ -f /root/shared/boinc_app ]; then
    cd /root/shared
    ./boinc_app
    shutdown -hP 0
else
    echo --- Failed to launch script
    sleep 5
fi
shutdown -hP 0
}}}

=== How it works ===

Using the example startup script, the steps in running a vboxwrapper app are:

 1. BOINC client
  * Create slot directory, say `slot/0`
  * Create '''slot/0/shared''', and copy input files there
  * Execute '''vboxwrapper''' in the slot directory
 1. vboxwrapper
  * Create and run virtual machine
 1. Virtual machine
  * Startup script
   * mounts '''shared''' directory
   * cd into shared directory
   * execute boinc_app
   * when boinc_app exits, shut down virtual machine
 1. vboxwrapper
  * delete virtual machine
  * call boinc_finish()
 1. BOINC client
  * copy output files from `slot/0/shared` to project directory

=== Creating base VM images ===

The VM image that you distribute need contain only the
runtime environment for your applications.
In particular, it need not contain:

 * Development tools such as gcc
 * GUI software such as X11, gtk etc.

Reducing the VM image size reduces the network load
on your server and on volunteer hosts, and the disk usage on volunteer hosts.

The easiest way to make a "small" Linux VM is to install the network install of Debian within the VM.
You can find the netinst images [http://www.debian.org/distrib/netinst here].
Such VMs have !VirtualBox and guest additions installed by default.
They have the runtime libraries needed to run C and C++ applications.

==== Role Selection ====

During install you'll be asked what role should this Linux machine be configured for.
Make sure all roles are unselected before continuing.

==== Cleaning the Debian VM ====

First step is to remove non-critical packages that are essential ([https://wiki.debian.org/ReduceDebian source]). For Debian Squeeze 7.1 that are:
{{{
acpi acpid busybox eject iamerican ibritish info ispell laptop-detect logrotate manpages net-tools os-prober tasksel traceroute usbutils wamerican
}}}
Use aptitude to also remove `linux-headers-*` packages but don't remove `dkms` or `virtualbox-guest-dkms`! Run `apt-get autoremove` after this to get rid of now unused packages. `apt-get autoclean` and `apt-get clean` should remove downloaded archives to free up space.

In order to realy purge all files from previously removed packages this command is helpfull as it purges all configuration files from uninstalled packages:
{{{
dpkg --purge `dpkg --get-selections | grep deinstall | cut -f1`
}}}

==== Updating Grub ====

If you want to speed up the boot process,
change the default timeout for grub by modifying `/etc/default/grub`:
{{{
GRUB_TIMEOUT = 0
}}}

After saving the update run:
{{{
update-grub
}}}

==== Updating Inittab ====

To configure Linux for automatic login you'll need to install
a different terminal handler.
mingetty works well for our purposes.

To install mingetty:
{{{
root@boinc-vm-image:/etc/default# apt-get install mingetty
}}}

Next you'll need to change the terminal handler assigned to the first virtual terminal by modifying `/etc/inittab`.
Change line:
{{{
1:2345:respawn:/sbin/getty 38400 tty1
}}}

To:
{{{
1:2345:respawn:/sbin/mingetty --autologin root --noclear tty1
}}}

==== Disabling periodic fsck ====

This will disable the periodic fsck run that occurs every 30 times the VM is turned on: `tune2fs -c -1 /dev/sda1`.

==== Compact the VDI container ====

To get a better compression ratio you may install and run the '''zerofree''' tool to overwrite all empty space on the VDI with zeros.
{{{
apt-get install zerofree
telinit 1
{enter root password}
mount -o remount,ro /dev/sda1
zerofree -v /dev/sda1
shutdown -hP 0
}}}

Now open a terminal on the host System and compact the VDI file:
{{{
vboxmanage modifyhd FILENAME.vdi --compact
}}}