Client data model

This document describes proposed changes to the client so that it can handle distributed storage.

Current

FILE_INFO elements

• generated_locally
• upload_when_present
• uploaded
• sticky
• urls

status (present, not present, error)

Problems

Proposed

FILE_INFO elements:

• upload_urls
• download_urls
• status

<file_info> elements in scheduler replies:

• if referenced from an app version or workunit, set download_if_not_present

• if referenced from a result, and no_upload is not present, set upload_when_present.

If we get an upuload request and the file isn't there, return error

Edge case: an input file that may have been generated by a previous job. If not, this job should generate it. Include such a file as an output, not an input. Input files have the property that if they're not present, we download them.