Changes between Initial Version and Version 1 of JobSubmission


Ignore:
Timestamp:
Sep 18, 2009, 3:57:58 PM (15 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • JobSubmission

    v1 v1  
     1[[PageOutline]]
     2= Submitting jobs =
     3
     4To submit a job you must
     5
     6 1. Write XML 'template files' that describe the job's input and outputs (typically the same template files can be used for many jobs).
     7 1. Create the job's input file(s), and put them in the right place (as determined by the file name) in the [DirHierarchy download directory hierarchy].
     8 1. Invoke a BOINC function or script that submits the job.
     9
     10Once this is done, BOINC takes over: it creates one or more instances of the job,
     11distributes them to client hosts, collects the output files,
     12finds a canonical instance, assimilates the canonical instance, and deletes files.
     13
     14Typically, steps 2) and 3) are done by a [WorkGeneration work generator] program
     15that creates lots of jobs.
     16
     17== Input and output template files == #templates
     18
     19An input template file has the form
     20{{{
     21<file_info>
     22    <number>0</number>
     23    [ <sticky/>, other attributes]
     24</file_info>
     25[ ... ]
     26<workunit>
     27    <file_ref>
     28        <file_number>0</file_number>
     29        <open_name>NAME</open_name>
     30    </file_ref>
     31    [ ... ]
     32    [ <command_line>-flags xyz</command_line> ]
     33    [ <rsc_fpops_est>x</rsc_fpops_est> ]
     34    [ <rsc_fpops_bound>x</rsc_fpops_bound> ]
     35    [ <rsc_memory_bound>x</rsc_memory_bound> ]
     36    [ <rsc_disk_bound>x</rsc_disk_bound> ]
     37    [ <delay_bound>x</delay_bound> ]
     38    [ <min_quorum>x</min_quorum> ]
     39    [ <target_nresults>x</target_nresults> ]
     40    [ <max_error_results>x</max_error_results> ]
     41    [ <max_total_results>x</max_total_results> ]
     42    [ <max_success_results>x</max_success_results> ]
     43    [ <credit>X</credit> ]
     44</workunit>
     45}}}
     46Elements and tags must be on separate lines as shown.
     47The components are:
     48
     49 <file_info>, <file_ref>::
     50
     51        Each pair describes an [BoincFiles#Fileproperties input file] and [BoincFiles#Filereferences the way it's referenced].
     52
     53 <command_line>::
     54
     55        The command-line arguments to be passed to the main program.
     56
     57 <credit>::
     58
     59        The amount of credit to be granted for successful completion of this workunit. Use this only if you know in advance how many FLOPs it will take. Your [ValidationSimple validator] must use get_credit_from_wu() as its compute_granted_credit() function.
     60
     61 Other elements::
     62        [JobIn Work unit attributes]
     63
     64An output template file has the form
     65{{{
     66<file_info>
     67    <name><OUTFILE_0/></name>
     68    <generated_locally/>
     69    <upload_when_present/>
     70    <max_nbytes>32768</max_nbytes>
     71    <url><UPLOAD_URL/></url>
     72</file_info>
     73<result>
     74    <file_ref>
     75        <file_name><OUTFILE_0/></file_name>
     76        <open_name>result.sah</open_name>
     77        [ <optional>0|1</optional> ]
     78        [ <no_validate>0|1</no_validate> ]
     79    </file_ref>
     80</result>
     81}}}
     82
     83Elements and tags must be on separate lines as shown.
     84Each <file_info>, <file_ref> pair describes an output file.
     85The elements include:
     86 '''<name>''' and '''<file_name>''':: the physical file name.  Typically use <OUTFILE_0>, <OUTFILE_1> etc.; this will be replaced with a generated name based on the job name.
     87 '''<generated_locally/>''':: always include this for output files.
     88 '''<max_nbytes>''':: maximum file size.  If the actual size exceeds this, the file will not be uploaded, and the job will be marked as an error.
     89 '''<url>''':: the URL of the file upload handler.  You may include this explicitly, or use '''<UPLOAD_URL/>''' to use the URL in your project's config.xml file.
     90 '''<open_name>''':: the "logical name" by which your application will reference the file.
     91 '''<optional>''':: if false, your application must create the file, otherwise the job will be marked as an error.
     92 '''<no_validate>''':: if true, don't include this file in the result validation process (relevant only if you are using the sample bitwise validator).
     93
     94Both elements may have other properties, see BoincFiles for the full list.
     95
     96Note: when a job is created, the name of its output template file is stored in the database.
     97The file is read when instances of the job are created, which may happen days or weeks later.
     98Thus, editing an output template file can affect existing jobs.
     99If this is not desired, you must create a new output template file.
     100
     101You can safely remove a workunit template file after creating your last WU with it.
     102However, you can't delete a result template file until any WU that refers to it is completed
     103(i.e. no more replicas will be created).
     104
     105== Submitting a job manually == #creatework-tool
     106
     107To move an input file to the download directory, use
     108{{{
     109dir_hier_path filename
     110}}}
     111which prints the full pathname and creates the directory if needed (run this in the project's root directory). For example:
     112{{{
     113cp test_files/12ja04aa `bin/dir_hier_path 12ja04aa`
     114}}}
     115copies an input file from the test_files directory to the download directory hierarchy.
     116
     117To submit a job, run bin/create_work from the project root directory:
     118{{{
     119create_work [ arguments ] infile_1 ... infile_n
     120}}}
     121Mandatory arguments are:
     122    --appname name::
     123  application name
     124    --wu_name name::
     125        workunit name
     126    --wu_template filename::           
     127 WU template filename relative to project root; usually in templates/
     128    --result_template filename::
     129          result template filename, relative to project root; usually in templates/
     130Optional arguments are:
     131    --batch n::
     132    --priority n::
     133
     134The following [JobIn job parameters] may be passed in the input template,
     135or as command-line arguments to create_work,
     136or not passed at all, in which case defaults will be used.
     137
     138    --command_line "-flags foo"::
     139    --rsc_fpops_est x:: FLOPs estimate
     140    --rsc_fpops_bound x::
     141    --rsc_memory_bound x::
     142    --rsc_disk_bound x::
     143    --delay_bound x::
     144    --min_quorum x::
     145    --target_nresults x::
     146    --max_error_results x::
     147    --max_total_results x::
     148    --max_success_results x::
     149    --opaque N::
     150    --additional_xml 'x':: This can be used to supply, for example, <credit>12.4</credit>.
     151
     152== Submitting jobs from a C++ program == #cpp-workgen
     153
     154BOINC's library provides the functions:
     155{{{
     156// convert filename to path in download hierarchy
     157//
     158SCHED_CONFIG::download_path(
     159    const char* filename,
     160    char* path,
     161);
     162
     163// submit a job
     164//
     165int create_work(
     166    DB_WORKUNIT& wu,
     167    const char* wu_template,                  // contents, not path
     168    const char* result_template_filename,     // relative to project root
     169    const char* result_template_filepath,     // absolute or relative to current dir
     170    const char** infiles,                     // array of input file names
     171    int ninfiles
     172    SCHED_CONFIG&,
     173    const char* command_line = NULL,
     174    const char* additional_xml = NULL
     175);
     176}}}
     177The ''name'' and ''appid'' fields of the DB_WORKUNIT structure must always be initialized.
     178Other job parameters may be passed either in the DB_WORKUNIT structure or in the input template file (the latter has priority).
     179On a successful return, wu.id contains the database ID of the workunit.