Changes between Initial Version and Version 1 of ServerDirs


Ignore:
Timestamp:
Apr 24, 2007, 1:17:55 PM (17 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ServerDirs

    v1 v1  
     1= Server directory structure =
     2
     3The directory structure for a typical BOINC project looks like:
     4{{{
     5PROJECT/
     6    apps/
     7    bin/
     8    cgi-bin/
     9    log_HOSTNAME/
     10    pid_HOSTNAME/
     11    download/
     12    html/
     13        inc/
     14        ops/
     15        project/
     16        stats/
     17        user/
     18        user_profile/
     19    keys/
     20    upload/
     21}}}
     22where PROJECT is the name of the project and HOSTNAME is the server host. Each project directory contains:
     23
     24    * apps: application and core client executables
     25    * bin: server daemons and programs.
     26    * cgi-bin: CGI programs
     27    * log_HOSTNAME: log output
     28    * pid_HOSTNAME: lock files, pid files
     29    * download: storage for data server downloads.
     30    * html: PHP files for public and private web interfaces
     31    * keys: encryption keys
     32    * upload: storage for data server uploads.
     33
     34The upload and download directories may contain large numbers (millions) of files. For efficiency they are normally organized as a hierarchy of subdirectories.
     35
     36== Hierarchical upload/download directories ==
     37
     38The data server for a large project may store 100Ks or millions of files at any given point. If these files are stored in 'flat' directories (project/download and project/upload) the data server may spend a lot of CPU time searching directories. To avoid this, BOINC uses hierarchical upload/download directories. Each directory has a set of 1024 subdirectories, named 0 to 3ff. Files are hashed (based on their filename) into these directories.
     39
     40The hierarchy is used for input and output files only. Executables and other application version files are in the top level of the download directory.
     41
     42This affects your project-specific code in a couple of places. First, your work generator must put input files in the right directory before calling [WorkGeneration create_work()]. To do this, it can use the function
     43
     44{{{
     45int dir_hier_path(
     46    const char* filename, const char* root, int fanout, char* result,
     47    bool make_directory_if_needed=false
     48);
     49}}}
     50This takes a name of the input file and the absolute path of the root of the download hierarchy (typically the download_dir element from config.xml) and returns the absolute path of the file in the hierarchy. Generally make_directory_if_needed should be set to true: this creates a fanout directory if needed to accomodate a particular file.
     51
     52Secondly, your validator and assimilator should call
     53{{{
     54int get_output_file_path(RESULT const& result, string& path);
     55}}}
     56or
     57{{{
     58int get_output_file_paths(RESULT const& result, vector<string>& );
     59}}}
     60to get the paths of output files in the hierarchy.
     61
     62A couple of utility programs are available (run this in the project root directory):
     63{{{
     64dir_hier_move src_dir dst_dir fanout
     65dir_hier_path filename
     66}}}
     67dir_hier_move moves all files from src_dir (flat) into dst_dir (hierarchical with the given fanout). dir_hier_path, given a filename, prints the full pathname of that file in the hierarchy.