Changes between Initial Version and Version 1 of JobTemplates

Timestamp:

Feb 15, 2013, 2:23:18 PM (11 years ago)

Author:

davea

Comment:

--

JobTemplates

@@ +1 @@
+= Input and output templates =
+
+== Input template files ==
+
+An input template file has the form
+{{{
+
+<input_template>
+    <file_info>
+        <number>0</number>
+        [ <gzip/> ]
+        [ <sticky/> ]
+        [ <no_delete/> ]
+        [ <report_on_rpc/> ]
+        [ <url>...</url> ]
+        [ <url>...</url> ]
+        [ <md5_cksum>...</md5_cksum> ]
+        [ <nbytes>...</nbytes> ]
+    </file_info>
+    [ ... other files  ]
+    <workunit>
+        <file_ref>
+            <file_number>0</file_number>
+            <open_name>NAME</open_name>
+            [ <copy_file/> ]
+        </file_ref>
+        [ ... other files ]
+        [ <command_line>-flags xyz</command_line> ]
+        [ <rsc_fpops_est>x</rsc_fpops_est> ]
+        [ <rsc_fpops_bound>x</rsc_fpops_bound> ]
+        [ <rsc_memory_bound>x</rsc_memory_bound> ]
+        [ <rsc_disk_bound>x</rsc_disk_bound> ]
+        [ <delay_bound>x</delay_bound> ]
+        [ <min_quorum>x</min_quorum> ]
+        [ <target_nresults>x</target_nresults> ]
+        [ <max_error_results>x</max_error_results> ]
+        [ <max_total_results>x</max_total_results> ]
+        [ <max_success_results>x</max_success_results> ]
+    </workunit>
+</input_template>
+}}}
+Elements and tags must be on separate lines as shown.
+The components are:
+
+ '''<file_info>''':: describes an [BoincFiles#Fileproperties input file].
+  '''<number>''':: use 0, 1, ...
+  '''<gzip/>''':: transfer the file in gzipped (compressed) format to reduce network usage.  '''You must stage the file with the --gzip option (see below)'''.  Only 7.0+ clients can handle compressed transfers; older clients will get the file in uncompressed form.
+  '''<sticky/>''':: if present, the file remains on the client after job is finished.
+  '''<no_delete/>''':: if present, the file is not deleted from the server after job is completed.  Use this if the file is used as input to more than one job.
+  '''<report_on_rpc/>''':: if present, report file in each scheduler request (for sticky files)
+  '''<url>, <md5_cksum>, <nbytes>''':: used only for "non-local" input files (see below)
+
+ '''<file_ref>''':: describes [BoincFiles#Filereferences the way the file is referenced].
+  '''<file_number>''':: 0, 1, etc.
+  '''<open_name>''':: the logical name of the file
+  '''<copy_file>''':: if present, the file is copied into the job's slot directory
+
+ '''<command_line>''':: The command-line arguments to be passed to the main program.
+  Note: if you're using the [WrapperApp BOINC wrapper],
+  use <append_cmdline_args/> in your job.xml file to pass command-line arguments from the wrapper
+  to the wrapped application.
+
+ '''<rsc_fpops_est>''' etc.::
+   [JobIn Job attributes] such has how much disk space will be used.
+   BOINC will supply reasonable defaults for these,
+   but you should supply the correct values;
+   otherwise, for example, BOINC might try to run the job
+   on a host with insufficient disk space.
+
+Notes:
+ * The input template is copied into a BLOB column of the workunit table and will substitute filenames,
+  and have download urls, signatures and other elements inserted into your provided template.
+  The total expanded BLOB cannot exceed 65,535 bytes.
+  This is enough for about 200 input files,
+  fewer if you use long file names are multiple download URLs.
+  If this isn't enough, you can use [FileCompression BOINC file compression] to zip several files into a single file reference for download,
+  and expanding them prior to running on the client machine.
+
+== Output template files ==
+
+An output template file has the form
+{{{
+<output_template>
+    <file_info>
+        <name><OUTFILE_0/></name>
+        <generated_locally/>
+        <upload_when_present/>
+        <max_nbytes>32768</max_nbytes>
+        <url><UPLOAD_URL/></url>
+    </file_info>
+    <result>
+        <file_ref>
+            <file_name><OUTFILE_0/></file_name>
+            <open_name>result.sah</open_name>
+            [ <copy_file>0|1</copy_file> ]
+            [ <optional>0|1</optional> ]
+            [ <no_validate>0|1</no_validate> ]
+        </file_ref>
+        [ <report_immediately/> ]
+    </result>
+</output_template>
+}}}
+
+Elements and tags must be on separate lines as shown.
+The elements include:
+
+ '''<file_info>''':: describes an output file.
+ '''<name>''':: the physical file name.
+  Typically use <OUTFILE_0>, <OUTFILE_1> etc.;
+  BOINC will replace this with a generated name based on the job name.
+
+ '''<file_ref>''':: describes how an output file will be referenced by the application.
+ '''<open_name>''':: the "logical name" by which the application will reference the file.
+ '''<copy_file/>''':: if present, the file will be generated in the slot directory,
+   and copied to the project directory after the job has finished.
+   Use this for [WrapperApp legacy applications].
+ '''<generated_locally/>''':: always include this for output files.
+ '''<max_nbytes>''':: maximum file size.
+  If the actual size exceeds this, the file will not be uploaded,
+  and the job will be marked as an error.
+ '''<url>''':: the URL of the file upload handler.
+  You may include this explicitly, or use '''<UPLOAD_URL/>'''
+  to use the URL in your project's config.xml file.
+ '''<optional>''':: if 0 or absent, your application must create the file,
+  otherwise the job will be marked as an error.
+ '''<no_validate>''':: if true, don't include this file in the result validation process
+  (relevant only if you are using the sample bitwise validator).
+ '''<no_delete/>''':: if present, the file will not be deleted on the server
+  even after the job is finished.
+
+ '''<report_immediately/>''':: if present, clients will report this job
+  immediately after the output files are uploaded.
+  Otherwise they may wait up to a day.
+  (Implemented in 6.12.27+ clients only).
+
+Note: when a job is created, the name of its output template file is stored in the database.
+The file is read when instances of the job are created, which may happen days or weeks later.
+Thus, editing an output template file can affect existing jobs.
+If this is not desired, you must create a new output template file.
+
+You can safely remove an input template file after creating your last job with it.
+However, output template files must exist until any task that refers to it is completed
+(i.e. no more replicas will be created).
+
+In general, you should not attempt to use more then 50 files in your output template.
+This is because the provided template will have upload URLs, and file names expanded, and signatures added to it.
+The total size of the result must remain under 65,535 bytes to fit in the xml_doc_in and xml_doc_out BLOBs in the result table.
+Your actual limit on number of files will depend upon the length of your job names, and upload URLs.
+If your expanded output template is approaching the size limit,
+you can use [FileCompression BOINC file compression] to zip several files
+into a single file reference for upload, prior to completing each task on the client machine.
+Once you have run some jobs through your project,
+you can compare the size of the expanded xml with the 65,535 limit by running the following MySQL statement:[[BR]]
+{{{select max(length(xml_doc_in)), max(length(xml_doc_out)) from result;}}}[[BR]]
+Note that here "_in" and "_out" are both references to the output template.
+"_in" being when the task was created, and "_out" how it looked,
+with actual file sizes and checksums, at the time the task was returned.
+So don't confuse these column names with the "input" and "output" templates.