Changes between Version 25 and Version 26 of CreditNew


Ignore:
Timestamp:
Mar 25, 2010, 10:17:58 AM (14 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • CreditNew

    v25 v26  
    3434
    3535 * Project neutrality: different projects should grant
    36    about the same amount of credit per day for a given processor.
     36   about the same amount of credit per host,
     37   averaged over all hosts.
     38
     39 * Cheat-proof
    3740
    3841It's easy to show that both goals can't be satisfied simultaneously.
     
    412415=== Database changes ===
    413416
    414 New table '''host_app''':
     417New table '''host_app_version''':
    415418{{{
    416419int    host_id;
    417 int    app_id;
    418 int    vnpfc_n;
    419 double vnpfc_sum;
    420 double vnpfc_exp_avg;
    421 }}}
    422 
    423 New table '''host_app_version''':
    424 {{{
    425 int    host_id;
    426 int    app_version_id;
    427 int    et_n;
    428 double et_sum;
    429 double et_exp_avg;
    430 // some variable for recent error rate,
    431 // replacing host.error_rate and host.max_results_day
    432 // make sure that a host w/ 1 good and 1 bad GPU gets few GPU jobs
     420int    app_version_id;          // generalized for anon platform
     421AVERAGE pfc;
     422AVERAGE_VAR et;                         // elapsed time / wu.rsc_fpops_est
     423double host_scale_time;
     424bool scale_probation;
     425double error_rate;
     426int max_jobs_per_day;
     427int n_jobs_today;
     428AVERAGE_VAR turnaround;
    433429}}}
    434430
    435431New fields in '''app_version''':
    436432{{{
    437 int    pfc_n;
    438 double pfc_sum;
    439 double pfc_exp_avg;
    440 double pfc_scaling_factor;
     433AVERAGE pfc;
     434double pfc_scale;
     435double expavg_credit;
     436double expavg_time;
    441437}}}
    442438
    443439New fields in '''app''':
    444440{{{
    445 int    vnpfc_n;
    446 double vnpfc_sum;
    447 double vnpfc_exp_avg;
    448 }}}
    449 
    450 === New request message fields ===
    451 
    452 === New reply message fields ===
     441double min_avg_pfc;
     442bool host_scale_check;          // whether to do scale probation
     443int max_jobs_in_progress;
     444int max_gpu_jobs_in_progress;
     445int max_jobs_per_rpc;
     446int max_jobs_per_day_init;
     447}}}
    453448
    454449=== Scheduler changes ===
    455450
    456 === Client changes ===
     451 * When dispatch anonymous app job,
     452   set result.app_version_id to -2/-3/-4 depending on resource.
     453 * update host_app_version.host_scale_time for
     454   app versions for which jobs are being sent
     455   and for which scale_probation is set.
    457456
    458457=== Validator changes ===
    459458
    460 === Server APIs for computing and granting credit ===
    461 
    462 == Compatibility ==
    463 
    464 
    465 
     459 * To reduce DB access, validator maintains a vector of app_versions.
     460   This is appended to by assign_credit_set().
     461   At the start of every validator pass, the pfc and expavg_credit
     462   fields of the app versions are saved.
     463   Updates are accumulated,
     464   and at the end of the validator pass (before sleep())
     465   the incremental changes are written to the DB.
     466   This scheme works correctly even with multiple validators per app.
     467 * The updating of app_versions is done in such a way that
     468   we pick up changes to pfc_scale by the feeder.
     469 * The app record is reread at the start of each scan,
     470   in case its min_avg_pfc has been changed by the feeder.
     471 * check_set() no longer returns credit (leave arg there for now)
     472 * update host_app_version.scale_probation in is_valid()
     473 * don't grant credit in is_valid()
     474 * compute and grant credit in handle_wu()
     475
     476=== Feeder changes ===
     477
     478 * If we're the "main feeder" (mod = 0, or mod not used),
     479   update app_version.pfc_scale and app.min_avg_pfc every 10 minutes.