Changes between Version 5 and Version 6 of AssimilateIntro


Ignore:
Timestamp:
Sep 17, 2009, 4:04:35 PM (15 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • AssimilateIntro

    v5 v6  
    1 = Result assimilation =
     1= Handling completed jobs =
    22
    3 Projects must create one assimilator program per application. This is done by linking the program '''sched/assimilator.cpp''' with an application-specific function of the form
     3Completed jobs are handled by programs called '''assimilators'''.
     4These are generally application-specific:
     5they might copy output files from the BOINC upload directory to a permanent location,
     6or they might parse the output files and insert results into a database.
     7
     8== Creating an assimilator ==
     9
     10To create an assimilator, link the program '''sched/assimilator.cpp''' with a function of the form
    411{{{
    512int assimilate_handler(
     
    815}}}
    916
    10 This is called when either
    11  * The workunit has a nonzero [JobIn#Redundancyandschedulingattributes error mask] (indicating, for example, too many error results). In this case the handler might write a message to a log or send an email to the application developer.
    12  * The workunit has a canonical result. In this case wu.canonical_resultid will be nonzero, canonical_result will contain the canonical result. Your handler might, for example, parse the canonical result's output file and write its contents to a separate database.
     17This function is called when either
     18 * The workunit has a nonzero [JobIn#Redundancyandschedulingattributes error mask] (indicating, for example, too many error results). In this case the handler might write a message to a log or send an email to the project administrator.
     19 * The workunit has a canonical result. In this case wu.canonical_resultid will be nonzero, and canonical_result will contain the canonical result.
    1320
    14 In both cases the 'results' vector will be populated with all the workunit's results (including unsuccessful and unsent ones). All files (both input and output) will generally be on disk.
     21In both cases the 'results' vector will be populated with all the workunit's results (including unsuccessful and unsent ones).
     22All files (both input and output) will generally be on disk.
    1523
    1624It's possible that both conditions might hold.
    1725
    1826The return values of `assimilate_handler()` are:
    19  * 0: the workunit will be marked as assimilated.
    20  * DEFER_ASSIMILATION: the workunit will not be marked as assimilated, and will be processed again when the next instance finishes.  This is useful for appliations where you want to see all the completed results.
    21  * Other nonzero values: the assimilator will log an error message and exit. Typically the function should return nonzero for any error condition. This way the system administrator can fix the problem before any completed or erroneous workunits are mis-handled by BOINC.
     27 * 0: success: the workunit will be marked as assimilated.
     28 * DEFER_ASSIMILATION: the workunit will be processed again when another instance finishes.  This is useful for appliations where you want to see all the completed results.
     29 * Other nonzero values: the assimilator will log an error message and exit. Typically '''assimilate_handler()''' should return nonzero for any error condition. This way the system administrator can fix the problem before any completed or erroneous workunits are mis-handled by BOINC.
    2230
    2331You can use BOINC's [BackendUtilities back-end utility functions] to get file pathnames and open files.
     32
     33== Running assimilators ==
     34
     35Run assimilators as BOINC daemons: that is, add an entry
     36{{{
     37<daemon>
     38   <cmd> my_assimilator -app APPNAME </cmd>
     39</daemon>
     40}}}
     41to your project's [ProjectDaemons configuration file].
     42
     43Assimilators have the followiong command-line options:
     44
     45 -app name :: the application name
     46
     47 [ -mod N R ] :: process only jobs with mod(ID, N) == R
     48 
     49 [ -d N ] :: set verbosity level (1 = least, 3 = most)
     50
     51 [ -dont_update_db ] :: don't mark jobs as assimilated (for testing)
     52 
     53== The sample assimilator ==
     54
     55BOINC includes a sample assimilator, '''sample_assimilator'''.
     56It does the following:
     57
     58 * For successful workunits, it writes the canonical instance's output files to the directory '''PROJECT/sample_results/'''.  If there is only one output file it is named WU_NAME.  If there are more than one they are named WU_NAME_0, WU_NAME_1, etc.  If there are no output files, an empty file WU_NAME_no_output_files is created.
     59 * If the workunit failed (to many errors, etc.) it appends a line to '''sample_results/errors''' containing the workunit name and the error code.
     60
     61The sample assimilator can be used as a placeholder while you
     62are developing your application.
     63In some cases you may be able to use it in production.