Changes between Version 4 and Version 5 of JobSubmission


Ignore:
Timestamp:
Sep 22, 2009, 9:31:56 AM (11 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • JobSubmission

    v4 v5  
    134134(i.e. no more replicas will be created).
    135135
    136 == Putting input files in the download directory ==
    137 
    138 To move an input file to the download directory, use
    139 {{{
    140 dir_hier_path filename
    141 }}}
    142 which prints the full pathname and creates the directory if needed (run this in the project's root directory).
    143 For example:
    144 {{{
    145 cp test_files/12ja04aa `bin/dir_hier_path 12ja04aa`
    146 }}}
    147 copies an input file from the test_files/ directory to the download directory hierarchy.
    148 
    149 [[PageOutline]]
    150 = Submitting jobs =
    151 
    152 To submit a job you must
    153 
    154  1. Write XML 'template files' that describe the job's input and outputs (typically the same template files can be used for many jobs).
    155  1. Create the job's input file(s), and put them in the right place (as determined by the file name) in the [DirHierarchy download directory hierarchy].
    156  1. Invoke a BOINC function or script that submits the job.
    157 
    158 Once this is done, BOINC takes over: it creates one or more instances of the job,
    159 distributes them to client hosts, collects the output files.
    160 It [ValidateIntro validates] and
    161 [AssimilateIntro processes] the results,
    162 and deletes the input and output files.
    163 
    164 Typically, steps 2) and 3) are done by a [WorkGeneration work generator] program
    165 that creates lots of jobs.
    166 
    167 == Input and output template files == #templates
    168 
    169 An input template file has the form
    170 {{{
    171 <file_info>
    172     <number>0</number>
    173     [ <sticky/> ]
    174     [ <nodelete/> ]
    175     [ <report_on_rpc/> ]
    176 </file_info>
    177 [ ... other files  ]
    178 <workunit>
    179     <file_ref>
    180         <file_number>0</file_number>
    181         <open_name>NAME</open_name>
    182         [ <copy_file/> ]
    183     </file_ref>
    184     [ ... other files ]
    185     [ <command_line>-flags xyz</command_line> ]
    186     [ <rsc_fpops_est>x</rsc_fpops_est> ]
    187     [ <rsc_fpops_bound>x</rsc_fpops_bound> ]
    188     [ <rsc_memory_bound>x</rsc_memory_bound> ]
    189     [ <rsc_disk_bound>x</rsc_disk_bound> ]
    190     [ <delay_bound>x</delay_bound> ]
    191     [ <min_quorum>x</min_quorum> ]
    192     [ <target_nresults>x</target_nresults> ]
    193     [ <max_error_results>x</max_error_results> ]
    194     [ <max_total_results>x</max_total_results> ]
    195     [ <max_success_results>x</max_success_results> ]
    196     [ <credit>X</credit> ]
    197 </workunit>
    198 }}}
    199 Elements and tags must be on separate lines as shown.
    200 The components are:
    201 
    202  '''<file_info>''':: describes an [BoincFiles#Fileproperties input file].
    203   '''<open_name>''':: the physical name of the file.
    204   '''<sticky/>''':: if present, the file remains on the client after job is finished.
    205   '''<nodelete/>''':: if present, the file is not deleted from the server after job is completed.
    206   '''<report_on_rpc/>''':: if present, report file in each scheduler request (sticky files)
    207 
    208  '''<file_ref>''':: describes [BoincFiles#Filereferences the way the file is referenced].
    209   '''<open_name>''':: the logical name of the file
    210   '''<copy_file>''':: if present, the file is copied into the job's slot directory
    211 
    212  '''<command_line>''':: The command-line arguments to be passed to the main program.
    213 
    214  '''<credit>'''::
    215         The amount of credit to be granted for successful completion of this workunit.
    216         Use this only if you know in advance how many FLOPs it will take.
    217         Your [ValidationSimple validator] must use {{{get_credit_from_wu()}}}
    218         as its compute_granted_credit() function.
    219 
    220  '''<rsc_fpops_est>''' etc.::
    221         [JobIn Work unit attributes]
    222 
    223 An output template file has the form
    224 {{{
    225 <file_info>
    226     <name><OUTFILE_0/></name>
    227     <generated_locally/>
    228     <upload_when_present/>
    229     <max_nbytes>32768</max_nbytes>
    230     <url><UPLOAD_URL/></url>
    231 </file_info>
    232 <result>
    233     <file_ref>
    234         <file_name><OUTFILE_0/></file_name>
    235         <open_name>result.sah</open_name>
    236         [ <optional>0|1</optional> ]
    237         [ <no_validate>0|1</no_validate> ]
    238     </file_ref>
    239 </result>
    240 }}}
    241 
    242 Elements and tags must be on separate lines as shown.
    243 The elements include:
    244 
    245  '''<file_info>''':: describes an output file.
    246  '''<name>''':: the physical file name.
    247   Typically use <OUTFILE_0>, <OUTFILE_1> etc.;
    248   this will be replaced with a generated name based on the job name.
    249 
    250  '''<file_ref>''' describes an the output file will be referenced by the application.
    251  '''<open_name>''':: the "logical name" by which your application will reference the file.
    252  '''<generated_locally/>''':: always include this for output files.
    253  '''<max_nbytes>''':: maximum file size.
    254    If the actual size exceeds this, the file will not be uploaded,
    255    and the job will be marked as an error.
    256  '''<url>''':: the URL of the file upload handler.
    257   You may include this explicitly, or use '''<UPLOAD_URL/>'''
    258   to use the URL in your project's config.xml file.
    259  '''<optional>''':: if 0 or absent, your application must create the file,
    260   otherwise the job will be marked as an error.
    261  '''<no_validate>''':: if true, don't include this file in the result validation process
    262   (relevant only if you are using the sample bitwise validator).
    263 
    264 Note: when a job is created, the name of its output template file is stored in the database.
    265 The file is read when instances of the job are created, which may happen days or weeks later.
    266 Thus, editing an output template file can affect existing jobs.
    267 If this is not desired, you must create a new output template file.
    268 
    269 You can safely remove a workunit template file after creating your last WU with it.
    270 However, you can't delete a result template file until any WU that refers to it is completed
    271 (i.e. no more replicas will be created).
    272 
    273136== Staging input files ==
    274137