wiki:PythonApps

Version 4 (modified by jeremy.cowles, 9 years ago) (diff)

--

Python applications

Python applications can be run two different ways: by using the PyMW framework (cross platform) or by using py2exe (Windows only). Both methods are discussed here.

Using PyMW

The PyMW framework is a master-worker computing framework for Python. Using PyMW in conjunction with a special interpreter (PyBOINC) allows you to send Python scripts as work units and does not require the client to have Python installed. PyMW can also be used without the portable Python interpreter, however, the client is responsible for installing the correct version of Python (2.4 or greater) and adding the interpreter to their system PATH environment variable.

In addition to BOINC, PyMW also provides non-BOINC interfaces for Condor, MPI, and simple multi-core processing which can be selected using a command line switch. The multi-core interface is particularly useful for debugging BOINC applications before submitting them to your project.

The following is a brief explanation of how to setup PyMW using the BOINC virtual server.

Installing and Testing PyMW

First, setup the BOINC virtual server following the instructions in the Quick Start guide. Also, if you are following this exactly, it's highly recommended that you ssh into your VM instead of using a VM viewer so that you can copy and paste commands.

First, login to your VM as "boincadm" (with password "boincadmpw" if you haven't changed it). To get up and running with PyMW, first download the PyMW package and unzip it into your home folder on the BOINC VM and install it into your local Python "site-packages" directory. This can be done with the following commmands, assuming you have internet access:

$ cd ~
$ wget http://downloads.sourceforge.net/project/pymw/pymw/PyMW%200.2/pymw-0.2.tar.gz
$ tar -xzf pymw-0.2.tar.gz
$ rm pymw-0.2.tar.gz
$ cd pymw-0.2
$ su -c "python setup.py install"

(the password for root is rootpw, unless you changed it)

Note that the URL for PyMW was current as of the time of this writing (v0.2), but you should check for and use the newest available release.

At this point, PyMW is installed and so it's a good idea to test the installation. While still in the PyMW directory that you extracted above, try the following example distributed with PyMW:

$ cd examples
$ python monte_pi.py

(some output should appear)

$ cd ../../

After a moment, you should see some vague estimate of PI printed out to the screen. This means that PyMW is installed and working properly. By default, the example that was run above uses the "generic" interface and does not interact with the BOINC server.

Next, you should install PyBOINC to allow your PyMW apps to run without Python being installed on the BOINC client's machines. The following downloads the zip file, extracts it and moves it into a convenient location:

$ wget http://bitbucket.org/jeremycowles/pyboinc/downloads/pyboinc-0.01.zip
$ unzip pyboinc-0.01.zip
$ rm pyboinc-0.01.zip
$ mv pyboinc-0.01/bin-python26 pyboinc
$ rm -r pyboinc-0.01
$ rm pyboinc-0.01.zip

Now that PyMW and PyBOINC are setup, it's time to run a real BOINC application with PyMW.

$ cd ~/pymw-0.2/examples
$ python monte_pi.py -n 1 -i boinc -p /home/boincadm/projects/sandbox -c /home/boincadm/pyboinc -a 1.00_python26.zip

(lots of information will scroll out and then it will eventually wait)

The Monte Carlo PI estimation application is waiting for BOINC clients to process it's work units. So go back to your main PC (not the VM) and open up the BOINC client (install it if needed).

Once you have the BOINC manager open, click on "advanced view", when debugging it's best if you stay in this view. Next, click "Tools->Attach to project" from the main application menu. Click "Next" and then enter your project URL ("http://192.168.1.190/sandbox" if you are following this exactly) and then click next again.

Now that you are attached to the project, it should be listed in the project view. Click on the project and click the "update" button to force it to request work from the server. The PyBOINC interpreter will download and then a handful of tasks should appear. Once these tasks complete you can hit "update" on the project again to force the client to send them back to the server.

Back at the BOINC VM, you should now see the results of the computation.

Running PyMW Applications

Note: the following assumes that you have a folder in your home directory called "pyboinc" containing the Python 2.6 binaries and that you have installed PyMW as described in the previous section.

First, you must setup a project for testing, I will call mine "sandbox" and it will be run on the local address I have given my VM, 192.168.1.190, but you should use whatever address or domain name that is setup on your VM:

$ cd ~/boinc/tools
$ ./make_project --url_base http://192.168.1.190 sandbox

(answer yes to all questions/warnings)

This may take a while to generate the server keys. After it's completed, you should go into your project directory and add it to your cron jobs and to the Apache server:

$ cd ~/projects/sandbox
$ su -c 'cat sandbox.httpd.conf >> /etc/apache2/httpd.conf'
  (you'll be prompted for the root password, which is "rootpw")

$ su -c 'apache2ctl -k restart'
$ crontab sandbox.cronjob

Now the project is created, but there is no application installed (notice, we never called xadd or update_versions). Luckily, PyMW will do this for us. Now all we have to do is run our application with the boinc interface and point it to our home directory:

Using Py2Exe

py2exe allows you to make a standalone executable that envelops both the Python executable, and precompiled bytecode for your program. It is Windows-only.

I converted a python .py script into three files, a main executable, the C-run time dll msvcrt71.dll, and a third executable that is seemingly needed on Win9x. This runs on a system with no Python otherwise on it, with no path or whatever adjustments.

A Python binding of the BOINC API is described here.