= Volunteer data archival =

'''Volunteer data archival''' means using disk space on volunteered home computers
to store large data files.
This document describes the design of a system to
provide volunteer data archival on BOINC.
We assume the goals include:
 * Storing large (e.g. petabyte) files.
   Files may be thousands of times larger than the
   amount of space available on individual computers.
 * Store files are long periods.
 * Be able to reduce the probability of data loss
   to arbitrarily small levels.

Properties of the volunteer host population include:

 * A host may be sporadically available because
   it is turned off, or because the user has suspended network activity.
   Unavailable periods may range from minutes to several days.
 * The upload and download speeds of hosts vary widely,
   and can be fairly low (e.g. 1 Mbps) in some cases.
 * The amount of disk space available to a project on a given host
   may fluctuate over time, because of the user's own disk usage
   or disk usage by other BOINC projects to which the host is attached.
 * The population is dynamic: hosts are constantly arriving and leaving.
   The mean lifetime of a host may be fairly small
   (on the order of 100 days).
 * Many hosts are behind firewalls.
   We assume that all communication is initiated by the BOINC client,
   and involves HTTP requests to trusted project servers.
   We don't consider direct client-to-client communication.

There are two basic techniques for achieving reliable storage using
unreliable resources:

 * '''Replication''': a file 

 * '''Coding''': with Reed-Solomon coding, a file is divided into N 'packets',
  and an additional K checksum packets are generated.
  The original data can be reconstructed from any N of these N+K packets.