Test framework for BOINC developers (not intended for general use)

BOINC logo
See the section on testing in Software Prerequisites. Note that a web server with PHP is required for running a real server, but that requires a lot of path and permissions configuration so we opt not to use them in testing.

Python testing framework

The test/ directory contains a library of Python modules that make end-to-end testing of BOINC easy. See the Python framework.

Quick start

Single test:
cd boinc/test && ./test_uc.py
Full test suite:
cd boinc/test && make check
After two or three minutes you should see 'Passed Test'. During the test you should see various progress status indicators (resize your window so that the status indicator line fits on one line).

Goals of the testing framework

The goal of the framework is to support automated testing of BOINC and BOINC applications. Each test is performed by a Python program that initializes the system to a deterministic state, executes the system, detects termination, and returns success or failure depending on the final state.

Many BOINC features involve multiple projects and/or multiple hosts. It would be prohibitively complex to do automated testing on multiple physical hosts. Fortunately the BOINC architecture makes it possible for multiple projects and (virtual) hosts to reside on a single physical host. So the testing framework uses this approach.

The framework lets you test systems with any of the following attributes:

The following system attributes are planned but not implemented yet:

TODO: Directory structure

TODO: Modules/ Classes

TODO: Example script

Test applications

The apps directory contains the following test applications:

Test cases

The test/ directory contains various python scripts named with prefix test_. Each of these is a test case which runs an end-to-end test of BOINC (creates directory structure for project(s) and host(s), starts a web server, server daemons, host client(s), and checks output is as expected).
test_uc.py The basic test using uppercase
test_concat.py tests command-line arguments and filenames
test_uc_slow.py tests checkpoint/restart mechanism
test_prefs.py tests some aspects of preferences.
test_water.py tests some aspects of water marks.
test_rsc.py tests that scheduling server only sends feasible work units.
test_pers.py tests the persistent file transfers for download and upload. It interrupts them in the middle and makes sure that the filesize never decreases along interrupted transfers.
test_masterurl_failure.py tests the exponential backoff mechanism on the client in case of master URL failures. This test is not automated. It has to be run, and then client.out (in the host directory) must be looked at to examine whether everything is working correctly.
test_sched_failure.py tests the exponential backoff mechanism on the client in case of scheduling server failures. This test is not automated. It has to be run, and then client.out (in the host directory) must be looked at to examine whether everything is working correctly.

Appendix: Optional Environment Variables

The following environment variables are optional and apply whatever web server you use:

BOINC_TEST_USER_NAME

User name to use for setting up database name. Defaults to $USER
BOINC_TEST_VERBOSE
Verbosity level.
0print nothing
1 [default]print some. If output is a tty, overwrite lines.
2print all
BOINC_TEST_DELETE
Specifies whether or not to delete testbed after the test finishes. Only relevant when BOINC_TEST_AUTO_SETUP=0. Possible values (case doesn't matter):
No
If-Successful [default]
Always
BOINC_TEST_INSTALL_METHOD
Specifies how to install html/php, CGI from source directories to testbed location.
link hardlink
symlink [default]symbolic link
copy copy
Copying is useful because it preserves what version of the file was used in a particular test run and hardlinking is best because compiled cgi and scheduler programs are not disrupted by parallel builds.

Appendix: Web Server

By default, the test script will use a custom web server that has no security and minimal cgi/php capability. You can also use Apache or some other web server with manually initialized directories by setting these environment variables: 
BOINC_TEST_AUTO_SETUP=0   [default=1]
BOINC_TEST_KEY_DIR
BOINC_TEST_PROJECTS_DIR
BOINC_TEST_CGI_URL
BOINC_TEST_HTML_URL
BOINC_TEST_CGI_DIR
BOINC_TEST_HTML_DIR
BOINC_TEST_HOSTS_DIR

Example setup

Bourne shell: 
    QHOME=/disks/philmor/a/users/quarl/proj
    TOP=$QHOME/test-boinc
    URL=http://milhouse.ssl.berkeley.edu/quarl

    export BOINC_TEST_PROJECTS_DIR=$TOP/projects
    export BOINC_TEST_USER_NAME=quarl
    export BOINC_TEST_SRC_DIR=$QHOME/boinc
    export BOINC_TEST_CGI_DIR=$TOP/boinc_cgi
    export BOINC_TEST_CGI_URL=$URL/boinc_cgi
    export BOINC_TEST_HTML_DIR=$TOP/boinc_html
    export BOINC_TEST_HTML_URL=$URL/boinc_html
    export BOINC_TEST_SHMEM_KEY=0x1717f00f
    export BOINC_TEST_KEY_DIR=$TOP/keys
    export BOINC_TEST_HOSTS_DIR=$TOP/host
Apache configuration: 
    <Directory /disks/philmor/a/users/quarl/proj/test-boinc>
            AllowOverride FileInfo AuthConfig Limit
            Options Indexes SymLinksIfOwnerMatch IncludesNoExec ExecCGI
        <Limit GET POST OPTIONS PROPFIND>
            Order allow,deny
            Allow from all
        </Limit>
    </Directory>

    ScriptAlias /quarl/boinc_cgi/ "/disks/philmor/a/users/quarl/proj/test-boinc/boinc_cgi/"
    Alias /quarl/ "/disks/philmor/a/users/quarl/proj/test-boinc/"
Return to BOINC main page

Last modified 9:09 PM UTC, April 20 2007.
Copyright © 2009 University of California. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.